Problems Widget Guidebook


Sphere Engine Problems is a service that allows you to manage programming problems and automatically verify solutions to these problems. Detailed information about the Sphere Engine Problems module can be found in the introduction.

Sphere Engine Problems Widget is a method of integration of the Sphere Engine Problems service via a component ready to be embedded on the website (widget). It is a quick and easy method of integration, recommended for the extension of existing projects. The alternative is the integration through API.

Embedding an interactive programming problem on the website has many applications. For example:

  • development of recruitment systems for automatic verification of programming competence,
  • extension of capabilities of the internal training platform for IT employees,
  • supplementing educational materials with exercises for independent solution,
  • automation of source code evaluation in programming competitions and hackathons.

Note: The above applications suggest the use of a widget in the extension of existing systems. Nothing stands in the way of using this integration method in building systems from scratch.

Important: The Sphere Engine Problems service can also be integrated with LMS systems. The integration method is - from the perspective of the end user - very similar. More information (including access to the demo version) can be found at the education.sphere-engine.com website.

Check out how it works

Below we present the embedded programming problem in the form in which it will be visible to end users after integration.

Widget integration

Before the widget is integrated, it is necessary to create and configure it. In the widgets management panel the Create widget button is available which, when pressed, initiates the creation process.

Configuration data required for integration is available in the Menu > Problems > Widgets > WIDGET NAME > Get the code.

JavaScript

It is required to attach two HTML snippets to your system. The first one is responsible for the initialization, the second is responsible for displaying the widget.

Initialization

The code responsible for initialization should be embedded only once (regardless of the number of widgets displayed on the site), right after the <body> element.

<script>(function(d, s, id){
  SE_BASE = "[customized_link]";
  SE_HTTPS = true;
  SE = window.SE || (window.SE = []);
  var js, fjs = d.getElementsByTagName(s)[0];
  if (d.getElementById(id)) return;
  js = d.createElement(s); js.id = id;
  js.src = (SE_HTTPS ? "https" : "http") + "://" + SE_BASE + "/static/sdk/sdk.min.js";
  fjs.parentNode.insertBefore(js, fjs);
}(document, "script", "sphere-engine-jssdk"));</script>
Display

The code responsible for displaying the widget has the following form:

<div class="se-widget" data-widget="[hash]"></div>

The [hash] parameter appearing as the value of the "data-widget" attribute is the unique identifier of the widget in the Sphere Engine system. You can find it in the Unique hash field in the Menu > Problems > Widgets > WIDGET NAME > Widget settings.

Displaying many widgets

The Sphere Engine Problems Widget allows you to embed multiple widgets on a single page. In the basic variant no additional action is required.

If you need to manage your widgets by JavaScript (see JavaScript SDK) it is necessary to identify them, which is done by means of a special identifier (marked as [id]) determined by the attribute data-id of the div element used to embed the widget. For example:

<div id="idInHTML" data-id="[id]" class="se-widget" data-widget="[hash]"></div>

Note: The widget is initialized automatically - immediately after loading the HTML document. Thanks to this it is possible to perform operations on the widget (e.g. configuration) before displaying it.

Note: The widget starts to load only when it becomes visible on the screen (i.e. when the user scrolls the page to the widget position). This allows you to speed up the page loading process which is especially useful when embedding multiple widgets.

Standalone page

An alternative to embed the widget directly on your site is to use the possibility of presenting it in the "standalone" mode, i.e. on a separate website.

The link to the page is generated when creating the widget. In the Menu > Problems > Widgets > WIDGET NAME > Get the code there is a ready-to-use link with the following structure:

https://[client_id].widgets.sphere-engine.com/lp?hash=[hash]

Note: The widget page in the "standalone" mode only displays the widget by default. In the Widget settings it is possible to define (in the form of HTML code) the header of the page with the widget. In the header, you can embed, for example, your own logo.

User session

The Sphere Engine Problems Widget provides user identification capabilities:

  • based on the se_user_id parameter (managed in the client's system),
  • based on cookies (within a internet browser).

User identification based on se_user_id has priority, i.e. if this parameter is specified, it will be used for identification.

Important: Use of the se_user_id parameter allows you to resume user session between different browsers, operating systems and even computers.

Security

The Sphere Engine Problem Widget provides the possibility of secure integration.

Checking the source

The use of security in the form of defining a list of web addresses, on which the widget can be placed minimizes the risk of unauthorized use of the widget on pages and sites outside of the list. It is the simplest and fastest method in configuration. It does not require the implementation of additional mechanisms.

The list of permissible addresses can be defined in Menu > Problems > Widgets > WIDGET NAME > Widget settings. In the configuration field Allow the widget to be used only at the following addresses you can specify one or multiple addresses, for example:

yoursite.com
yoursite.com/some/path
your-other-site.com
http://yoursite.com
https://yoursite.com

The Sphere Engine system verifies the origin of communication between the widget and the external system. Only requests from the list of defined addresses are accepted.

Warning: If you do not specify the list of addresses (i.e. leave it empty), the widget can be embedded on any page. This configuration should be used exclusively during integration work and tests.

Signature

In the production environment, we recommend using security with a signature (based on the dedicated easy to implement algorithm), which protects the widget from unauthorized use. The signature verification requirement must be enabled in advance by selecting the Secure the widget option in Menu > Problems > Widgets > WIDGET NAME > Widget settings.

The use of a signature for communication between the client system and the Sphere Engine increases the security level, i.e.:

  • eliminates the possibility of unauthorized access to the widget (preventing even complex and difficult access attempts performed by preparing false messages),
  • guarantees the reliability of transmitted data.
Signature parameters

Signature generation is based on selected widget parameters and additional signature parameters:

  • hash - the unique identifier of the widget in the Sphere Engine system. You can find it in the Unique hash field in Menu > Problems > Widgets > WIDGET NAME > Widget settings.
  • se_user_id - unique (for a given widget) end user ID of the widget (i.e. the persons solving the problem).
  • se_secret - secret key being the basis for the security of the signature. You can find it in the Shared secret field in Menu > Problems > Widgets > WIDGET NAME > Widget settings.
  • (optional) se_nonce - unique (for a given widget) session ID.

Important: The se_secret parameter it is the basis for the security of the signature and should remain secret. Do not send it in messages between systems.

Important: The signature generation procedure in the client system should not be performed on the front-end side of the application (e.g. by a function in JavaScript) due to the disclosure of the se_secret parameter.

Note: The use of the optional se_nonce parameter ensures that each end user session (i.e., every situation in which the widget has been initialized) is a one-off. It is not possible to re-initialize the widget using the se_nonce parameter with the same value. The use of this parameter provides full protection and security against unauthorized use of the widget.

Signature generation algorithm

The procedure for generating a signature can be divided into the following steps:

  • Normalization by sorting parameter names in alphabetical order. For example:
    [hash, se_user_id, se_secret, se_nonce] ➡ [hash, se_nonce, se_secret, se_user_id].
  • Conjunction by saving parameters and their values in the form of an inscription with the following format:
    param1=value1&param2=value2&...&paramN=valueN
    for example:
    hash=XYZ&se_nonce=12345&se_secret=CIPHER&se_user_id=bradpitt
  • (optional - only if there are non-valid characters in the ASCII parameter values) Encoding for the application/x-www-form-urlencoded format using the UTF-8 encoding.
  • Hashing using the SHA-256 algorithm which is available for most popular programming languages (in the standard library or as part of an additional library).

Pseudocode of the algorithm that generates the signature:

generate_signature(params) {
    // Normalization
    params = sort(params);

    // Conjunction with Encoding
    seq = [];
    foreach p in params {
        seq += key(p) + "=" + urlencode(value(p), "utf-8"); 
    }

    // Hashing
    return sha256(join("&", seq));
}

Implementation of algorithm that generates a signature in PHP:

function generate_signature($params) {
    ksort($params);

    $seq = [];
    foreach ($params as $k => $v) {
        $seq[] = $k . "=" . urlencode($v);
    }

    return hash("sha256", implode("&", $seq));
}

Implementation of algorithm that generates a signature in Python:

def generate_signature(params):
    keys = sorted(params.keys())

    seq = []
    for k in keys :
        seq.append(k + "=" + urllib.parse.quote_plus(params[k]))

    return hashlib.sha256("&".join(seq).encode('utf-8')).hexdigest()
Attaching a signature

Signature and values of all parameters used to generate it (except the se_secret parameter, which should remain secret) should be attached to the embedded widget in the form of the appropriate HTML attributes with the data- prefix:

  • data-widget for the hash parameter,
  • data-user-id for the se_user_id parameter,
  • data-nonce for the se_nonce parameter,
  • data-signature for the generated signature.

An example of HTML code used to embed a widget prepared to support the signature:

<div class="se-widget" 
    data-id="widget-secured-by-signature" 
    data-widget="XYZ"  
    data-signature="e85b12137f76526ca6804625fc4e2093a0750fc6c6c158fc2be210ac5" 
    data-user-id="bradpitt"
    data-nonce="e9838cc0819d713b3670df7adb0079a3">
</div>

For those interested: The general scheme for securing communication by signature assumes that both communicating pages share a secret key (so-called shared secret).
The sender uses the selected message parameters (or even the entire message) along with the key to create the so-called signature. The generated signature is attached to the message and sent to the recipient.
The recipient, who also has access to the key, repeats the procedure for creating the signature. The sent message is authorized, if the signature sent by the sender is identical to the signature generated by the recipient.

JavaScript SDK

The Sphere Engine JavaScript SDK library is loaded asynchronously. For this reason, before using the functions provided by the library, you must be sure that it has been initialized.

First of all, it is required to place the following code snipped right after the initialization of the Sphere Engine Problems Widget:

<script>SE.ready = function(f){if (document.readyState != "loading" && document.readyState != "interactive") f();else window.addEventListener("load", f);};</script>

The SE.ready() function has been designed in a way that ensures that functionalities using the JavaScript SDK will be used only when they are ready:

SE.ready(function() {
    // SDK functions usage
});

Use of the SE.ready() function makes sure that the library has been loaded, the SE object has been initialized and no errors will occur while parsing the scripts.

Access to the widget

Access to the widget is obtained by using the SE.widget() function, which based on the given widget ID (previously identified by the attribute data-id) returns the class object SEWidget used to manage the widget:

SE.ready(function() {
    var widget = SE.widget("widget");
    // variable "widget" is ready to use
});

The SEWidget class provides the following widget management capabilities:

  • access widgets,
  • dedicated events and event management.

Methods

The global SE object provides public methods for managing widgets.

SE.widget()
A SEWidget object is created that represents the widget.

Parameters

Name Type Description
id string ID of the widget placed in the HTML document

Returned value

The class SEWidget object is returned.

Example

Press the button to load widget
Load widget
<div data-id="example-widget" data-widget="{widget_hash}"></div>
<script>
    SE.ready(function() {
        $('#btn-load-widget').on('click', function(e) {
            e.preventDefault();
            var SEWidget = SE.widget("example-widget");
        });
    });
</script><a href="#" id="btn-load-widget">Load widget</a>

SEWidget.events.subscribe()
Assigns a function to the event (see events list).

Parameters

Name Type Description
event string the name of the event to assign a function
callback function function to be called on an event

Example

<div data-id="example-widget" data-widget="{widget_hash}"></div>
<script>
SE.ready(function() {

    var callback = function(data) {
        // Your code goes here
    };

    var SEWidget = SE.widget("example-widget");
    SEWidget.events.subscribe('{name_of_the_event}', callback);

});
</script>

SEWidget.events.unsubscribe()
Removes a function from the list of functions assigned to the event (see events list).

Parameters

Name Type Description
event string the name of the event to delete the function
callback function function to be removed from the list of functions at the event

Example

<div data-id="example-widget" data-widget="{widget_hash}"></div>
<script>
SE.ready(function() {

    var callback = function(data) {
        // Your code goes here
    };

    var SEWidget = SE.widget("example-widget");
    SEWidget.events.unsubscribe('{name_of_the_event}', callback);

});
</script>

Events

The SEWidget class object has a collection of dedicated events called at specified times of the widget operation.

beforeSendSubmission
The event is invoked before sending the request to the Sphere Engine system (i.e. after pressing the "Submit" button but before the actual sending of the submission).

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
submissionSource string the source code of the program sent in the submission
submissionLanguage string the programming language in which the program was implemented

The value returned by the callback function

The value returned by the callback function determines whether the submission will be sent to the Sphere Engine system:

  • true - the submission is to be sent
  • false - sending the submission will be stopped

Example

Data:
Submit submission to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>
<script>
    SE.ready(function() {

        var beforeSendSubmission = function(data) {

            $('#result').html('')
                .append("> data.submissionLanguage: " + data.submissionLanguage + "<br><br>")
                .append("> data.submissionSource: <br>" + data.submissionSource + "<br><br>");

            return true;
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('beforeSendSubmission', beforeSendSubmission);
    });
</script>

Data:
<pre id="result" style="height: 300px;">Submit submission to see the result</pre>

afterSendSubmission
The event is dispatched immediately after sending the submission to the Sphere Engine system.

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
submissionId integer report identifier in the Sphere Engine system

Example

Data:
Submit submission to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>
<script>
    SE.ready(function() {

        var afterSendSubmission = function(data) {

            $('#result').html('')
                .append("> data.submissionId: " + data.submissionId + "<br><br>");
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('afterSendSubmission', afterSendSubmission);
    });
</script>

Data:
<pre id="result" style="height: 100px;">Submit submission to see the result</pre>

checkStatus
The event is invoked regularly while checking the status of the submission during execution.

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
submissionId integer submission identifier in the Sphere Engine system
submissionTime integer start time in seconds (after startup)
submissionMemory integer memory consumption in kilobytes (after startup)
statusNumber integer the numeric value of the submission status (see: status list)
statusDescription string description of the submission status

Example

Data:
Submit submission to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>

<script>
    SE.ready(function() {

        var checkStatus = function(data) {
            $('#result')
                .append("<br><br>")
                .append("> submissionId: " + data.submissionId + "<br>")
                .append("> statusNumber: " + data.statusNumber + "<br>")
                .append("> statusDescription: " + data.statusDescription + "<br>")
                .append("> submissionMemory: " + data.submissionMemory + "<br>")
                .append("> submissionTime: " + data.submissionTime + "<br>");

            $('#result').scrollTop($('#result').prop("scrollHeight"));
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('checkStatus', checkStatus);
    });
</script>

Data:
<pre id="result" style="height: 300px;">Submit submission to see the result</pre>

timeOver
The event is triggered after the time set for resolving the problem has expired.

Example

Data:
Start the session and wait 1 minute
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>

<script>
    SE.ready(function() {
        var timeOver = function() {
            $('#result').html('the session is over');
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('timeOver', timeOver);
    });
</script>

Data:
<pre id="result" style="height: 50px;">Start the session and wait 1 minute</pre>

welcomeUser
The event is triggered after sending the completed username and email address on the welcome page (available only for the widget configuration with the welcome page enabled)).

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
userName string username provided on the welcome page
userEmail string email address provided on the welcome page

Example

Data:
Complete the form to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>

<script>
    SE.ready(function() {
        var welcomeUser = function(data) {
            $('#result').html('')
                .append("> userName: " + data.userName + "<br><br>")
                .append("> userEmail: " + data.userEmail + "<br><br>");
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('welcomeUser', welcomeUser);
    });
</script>

Data:
<pre id="result" style="height: 100px;">Complete the form to see the result</pre>

Receiving the result of the submission

Sphere Engine Problems Widget allows you to configure the mode in which results are sent directly to the client's system.

In Menu > Problems > Widgets > WIDGET NAME > Widget settings there is a field described as Callback URL. You can specify the exact return address in this field. The Sphere Engine system uses the address provided in this field to provide information about user submissions selected by them for evaluation.

Request specification

The Sphere Engine system sends a POST request to the callback address provided in Menu > Problems > Widgets > WIDGET NAME > Widget settings. The body of the request contains information about the submission and the user in JSON format (i.e. the request specifies a Content-type header with a application/json value).

The structure of the body of the request is as follows:

{
    message:
    {
        id: [messageId],
        name: [messageName]
    },
    body: {
        user: {
            uuid: [userUUID],
            userId: [userCustomId],
            userDefinedName: [userUserDefinedName],
            userDefinedEmail: [userUserDefinedEmail]
        }
        context: {
            hash: [widgetHash],
            name: [widgetName]
    }
        grade: {
            scoreInPoints: [scoreInPoints],
            scorePercentage: [scorePercentage]
        }
    }
}

Parameters appearing in the message section:

  • messageId - unique message identifier,
  • messageName - type of message (currently only "sendGrade").

Parameters appearing in the body.user section:

  • userUUID - the unique user identifier in the Sphere Engine system,
  • userCustomId - user identifier defined in the client system,
  • userUserDefinedName - username provided in the welcome form,
  • userUserDefinedEmail - email given in the welcome form.

Parameters appearing in the body.context section:

  • widgetHash - the unique identifier of the widget in the Sphere Engine system,
  • widgetName - name of the widget.

Parameters appearing in the body.grade section:

  • scoreInPoints - the number of points obtained for the submission,
  • scorePercentage - percentage ratio to the best submission.

Structure of the response

After sending the result of the submission, the Sphere Engine system waits for confirmation of receipt of messages on the client's system side.

A response is expected with the HTTP 200 code in JSON format (including the HTTP header Content-type: application / json) with the following structure:

{
    status: [receptionStatus]
}

The parameter receptionStatus can take one of two values:

  • success - in the event of successful receipt and processing of information about the result of the submission,
  • failure - in case of failure.

In the case of a negative response from the client's system (i.e. an HTTP response with code other than 200 or the "failure" value of the receptionStatus) Sphere Engine system will start to send information about the result of the submission in accordance with the schedule.

Notification schedule

In the event of a negative confirmation from the client's system, the Sphere Engine system will resubmit information about the result of the request.

Re-attempts to send information are made with a delay depending on the number of the try in accordance with the table below:

Try no. Delay
1 - 10 1 minute
11 - 20 5 minutes
21 - 30 10 minutes
31 - 40 30 minutes
41 - 50 1 hour
51 - 60 6 hours

Carrying out further attempts with increasing delay time secures the client's system against data loss during unavailability (e.g. during a failure). After 60 unsuccessful attempts, the Sphere Engine system stops sending information about the result of the submission.

Note: Carrying out all 60 attempts occupies a total of more than 75 hours. In most cases, this is sufficient time to perform repairs in the event of a client's system failure. In the event of unavailability of the system for a longer period of time, the resumption of information sending can be determined with technical support after prior contact at the following address: support@sphere-research.com.

Signing the message

The callback request sent by the Sphere Engine system is signed using the authorial signature algorithm (described in the section devoted to security). Validation of the signature on the client's system side is not mandatory, although it is recommended for security reasons.

Signature parameters

Signature verification requires taking into account the signature, widget parameters and additional signature parameters used to generate the signature:

  • hash - the unique identifier of the widget in the Sphere Engine system. You can find it in the Unique hash field in Menu > Problems > Widgets > WIDGET NAME > Widget settings,
  • se_body_hash - the effect of the SHA-1 hash function on the body of the return request,
  • se_nonce - unique message identifier,
  • se_secret - secret key that prevents signature falsification,
  • se_signature - signature.

Important: The se_secret parameter is the basis for the security of the signature and should remain secret. Do not send it in messages between systems.

Location of the parameters required to verify the signature:

Name of parameter Location or method of generation
hash included in the body of the back request (key: body.context.hash)
se_body_hash value of the SHA-1 hash function for the body of the back request
se_nonce the value of the HTTP SphereEngineNonce header name
se_secret available in the Shared secret field in Menu > Problems > Widgets > WIDGET NAME > Widget settings
se_signature the value of the HTTP SphereEngineSignature header named
Signature verification algorithm

The presented algorithm uses the signature generation algorithm discussed earlier.

verify_signature(headers, request_body) {
    var signature = headers["SphereEngineSignature"];

    var params = [
        "hash": request_body.body.context.hash,
        "se_body_hash": sha1(request_body),
        "se_nonce": headers["SphereEngineNonce"],
        "se_secret": CONFIG["SEPW_SHARED_SECRET"]
    ]

    return signature == generate_signature(params);
}

Message ID (nonce)
With each message, the system Sphere Engine transmits its unique identifier (se_nonce parameter) in order to secure the system against the repetition of the same message.

The decision to operate this parameter lies with the client. Checking the uniqueness of the se_nonce parameter is optional but is recommended for security reasons.