Problems Widget


Sphere Engine Problems Widget allows you to embed programming problems on a website. More precisely, it allows you to integrate the Problems module of the Sphere Engine service via a component known as the widget, that's ready to be embedded on a website. Sphere Engine Problems is a service that allows you to manage programming problems and automatically verify solutions to these problems. Detailed information about the Sphere Engine Problems module can be found in the introduction.

Embedding the widget is a quick and easy process allowing for the integration of any IT system with the Sphere Engine Problems module and recommended for both the expansion of existing projects and for projects built from scratch. An alternative method is integration using RESTful API.

The ability to place a programming problem that offers automatic verification of the submitted solutions on the website has many applications. For example:

  • extending recruitment systems with automatic verification of programming competences,
  • extending the internal training platform for IT employees,
  • supplementing educational materials with exercises to be solved by the students on their own,
  • automation of source code evaluation in programming contests and hackathons.

Important: Sphere Engine Problems can also be integrated with LMS. You can find more information (including access to a demo version) on education.sphere-engine.com.

See how it works

Below we present an embedded programming problem in the form in which it will be visible to end-users after integration.

Widget integration

Before the widget is integrated, it is necessary to create and configure it. In the widget's management panel (Menu > Problems > Widgets), you can find the Create widget button which, when pressed, initiates the configuration process.

JavaScript

Using this method, you should attach to your code two snippets of HTML code responsible for the initialization and displaying the widget.

Widget initialization

The code responsible for widget initialization should remain embedded only once (regardless of the number of widgets displayed on the page), immediately after the <body> element.

A personalized and ready-to-use widget initialization code can be found in Menu > Problems > Widgets > WIDGET NAME > Get the code:

<script>(function(d, s, id){
  SE_BASE = "<customized_link>";
  SE_HTTPS = true;
  SE = window.SE || (window.SE = []);
  var js, fjs = d.getElementsByTagName(s)[0];
  if (d.getElementById(id)) return;
  js = d.createElement(s); js.id = id;
  js.src = (SE_HTTPS ? "https" : "http") + "://" + SE_BASE + "/static/sdk/sdk.min.js";
  fjs.parentNode.insertBefore(js, fjs);
}(document, "script", "sphere-engine-jssdk"));</script>
Displaying the widget

The following code should be embedded on the page in the spot envisioned for the widget:

<div class="se-widget" data-widget="<hash>"></div>

The hash parameter which is the value of the data-widget attribute is the unique ID of the widget in the Sphere Engine system. It can be found in the Unique hash field in Menu > Problems > Widgets > WIDGET NAME > Widget settings.

Displaying multiple widgets

Sphere Engine Problems Widget allows you to embed multiple widgets on a single page. In the basic variant, no additional actions are required.

If you need to manipulate widget's behavior using JavaScript (see JavaScript SDK) it is necessary to identify them, which is done by means of a special identifier (marked as <id>) denoted by the data-id attribute of the div element used to embed the widget. For example:

<div id="idInHTML" data-id="<id>" class="se-widget" data-widget="<hash>"></div>

Note: The widget is initialized automatically - directly after the page loads in the browser - even if the widget is not yet visible on the page. This makes it possible to perform operations on the widget (e.g. configuration) before displaying it.

Note: The process of loading (and, as a result, displaying) the widget begins only when the widget becomes visible on the screen (i.e. when the user scrolls the page to the widget's position). Thanks to this, the loading of other elements of the page is not blocked. This allows for the page to load faster, which is especially useful when embedding multiple widgets.

Standalone page

An alternative way to embed the widget directly on your site is to use the feature of presenting it as a "standalone page", i.e. on a separate website in the sphere-engine.com domain.

A link to a standalone page is generated when creating the widget. You will find a ready-to-use link in Menu > Problems > Widgets > WIDGET NAME > Get the code. It has the following structure:

https://<client_id>.widgets.sphere-engine.com/lp?hash=<hash>

Note: The Standalone page section of the Menu > Problems > Widgets > WIDGET NAME > Get the code page provides a configuration field that allows you to define (in the form of HTML code) the header of the page with the widget. You can, for example, embed your own logo in the header.

User session

Sphere Engine Problems Widget provides user identification capabilities:

  • based on the se_user_id parameter (managed in the client's system),
  • based on cookies (in one browser).

User identification based on se_user_id has priority, i.e. if this parameter is specified, it will be used for identification instead of a cookie (even if it exists).

Important: Using the se_user_id parameter allows you to resume user sessions between different web browsers and even different devices.

Security

Sphere Engine Problems Widget provides mechanisms for secure integration. A fully secured widget can be embedded only on selected websites (i.e. on websites in the appropriate domain) and the communication between the client's system and the Sphere Engine system is fully reliable (i.e. unauthorized people are unable to falsify or fabricate communication).

Checking the source

The use of a security measure in the form of a list of defined web addresses on which a widget can be embedded minimizes the risk of unauthorized use of the widget on pages not on the list. It is a method that's the simplest and the fastest to configure. It doesn't require the implementation of additional mechanisms. It is recommended to use the signature mechanism to ensure full protection of the widget.

You can define the list of allowed addresses on the Menu > Problems > Widgets > WIDGET NAME > Widget settings page. In the Allow the widget to be used only at the following addresses configuration field, you can specify one or multiple addresses, for example:

yoursite.com
yoursite.com/some/path
your-other-site.com
http://yoursite.com
https://yoursite.com

The Sphere Engine system verifies HTTP requests from end-users using the client's system (i.e. the page where the widget is embedded). Only requests directed to addresses in the list of defined addresses are accepted.

Note: If you do not specify the list of addresses (i.e. leave it empty), the widget can be embedded on any page. This configuration should be used only during integration works and testing.

Signature

In a production environment, we recommend using the signature security measure that fully protects the widget from unauthorized use. The signature verification feature must be enabled in advance by selecting the Secure the widget option in Menu > Problems > Widgets > WIDGET NAME > Widget settings.

The use of a signature for communication between the client's system and the Sphere Engine system increases the security level, i.e .:

  • eliminates the possibility of unauthorized access to the widget (preventing even complex and difficult attempts at acquiring access by preparing fake HTTP requests)
  • guarantees the reliability of transmitted data.
Signature parameters

Signature generation is based on selected widget parameters and additional signature parameters:

  • hash - unique widget ID in the Sphere Engine system. It can be found in the Unique hash field in Menu > Problems > Widgets > WIDGET NAME > Widget settings.
  • se_user_id - unique (for a given widget) ID of the end-user of the widget (i.e. the person solving the task).
  • se_secret - secret key which is the basis for the security of the signature. It can be found in the Shared secret field in Menu > Problems > Widgets > WIDGET NAME > Widget settings.
  • (optional) se_nonce - unique (for a given widget) session ID.

Important: The se_secret parameter is the basis for the security of the signature and should remain a secret. Do not send it in messages between systems.

Important: The procedure of signature generation in the client's system should not be performed on the front-end side of the application (e.g. using a JavaScript function) due to the se_secret parameter being disclosed.

Note: The use of an optional se_nonce parameter ensures that each session of the end-user (i.e., every situation in which the widget has been initialized) is a one-off. It is impossible to re-initialize the widget using the se_nonce parameter with the same value. Using this parameter provides full protection against unauthorized use of the widget.

Signature generation algorithm

The signature generation procedure can be divided into the following steps:

  • Normalization by sorting parameter names in alphabetical order. For example:
    [hash, se_user_id, se_secret, se_nonce] ➡ [hash, se_nonce, se_secret, se_user_id].
  • Conjunction by saving the parameters and their values ​​in the form of a string with the following format:
    param1=value1&param2=value2&...&paramN=valueN
    for example:
    hash=XYZ&se_nonce=12345&se_secret=CIPHER&se_user_id=bradpitt
  • Encoding to the application/x-www-form-urlencoded format using UTF-8 encoding.
  • Hashing using the SHA-256 algorithm which is available for the majority of popular programming languages ​​(in the standard library or as part of an additional library).

Pseudocode of the algorithm that generates the signature

generate_signature(params) {
    // Normalization
    params = sort(params);

    // Conjunction with Encoding
    seq = [];
    foreach p in params {
        seq += key(p) + "=" + urlencode(value(p), "utf-8"); 
    }

    // Hashing
    return sha256(join("&", seq));
}

Implementation of the algorithm that generates the signature in PHP

function generate_signature($params) {
    ksort($params);

    $seq = [];
    foreach ($params as $k => $v) {
        $seq[] = $k . "=" . urlencode($v);
    }

    return hash("sha256", implode("&", $seq));
}

Implementation of the algorithm that generates the signature in Python

def generate_signature(params):
    keys = sorted(params.keys())

    seq = []
    for k in keys :
        seq.append(k + "=" + urllib.parse.quote_plus(params[k]))

    return hashlib.sha256("&".join(seq).encode('utf-8')).hexdigest()
Attaching a signature

The signature and the values ​​of all parameters used to generate it (except the se_secret parameter which should remain secret) should be attached to the embedded widget in the form of the appropriate HTML attributes with the data- prefix:

  • data-widget for the hash parameter,
  • data-user-id for the se_user_id parameter,
  • data-nonce for the se_nonce parameter,
  • date-signature for the signature.

An example of HTML code used to embed a widget prepared to support the signature

<div class="se-widget" 
    data-id="widget-secured-by-signature" 
    data-widget="XYZ"  
    data-signature="e85b12137f76526ca6804625fc4e2093a0750fc6c6c158fc2be210ac5" 
    data-user-id="bradpitt"
    data-nonce="e9838cc0819d713b3670df7adb0079a3">
</div>

For those interested: The general scheme for securing communication with the signature assumes that both communicating parties share a secret key (so-called shared secret).
The sender uses the selected parameters of the message (or even the entire message) along with the key to create the so-called signature. The generated signature is attached to the message and sent to recipient.
The recipient, who also has access to the key, repeats the procedure of creating the signature. The received message is authorized, if the signature sent by the sender is identical to the signature generated by the recipient.

JavaScript SDK

The Sphere Engine JavaScript SDK library is loaded asynchronously. For this reason, before using the functions provided by the library, we must be sure that it has been initialized.

First, place the following snippet just after the code that initializes the Sphere Engine Problems Widget:

<script>SE.ready = function(f){if (document.readyState != "loading" && document.readyState != "interactive") f();else window.addEventListener("load", f);};</script>

As a result, the initialization code should look as follows:

<script>(function(d, s, id){
  SE_BASE = "<customized_link>";
  SE_HTTPS = true;
  SE = window.SE || (window.SE = []);
  var js, fjs = d.getElementsByTagName(s)[0];
  if (d.getElementById(id)) return;
  js = d.createElement(s); js.id = id;
  js.src = (SE_HTTPS ? "https" : "http") + "://" + SE_BASE + "/static/sdk/sdk.min.js";
  fjs.parentNode.insertBefore(js, fjs);
}(document, "script", "sphere-engine-jssdk"));
SE.ready = function(f){if (document.readyState != "loading" && document.readyState != "interactive") f();else window.addEventListener("load", f);};
</script>

The above code defines the SE.ready() function which has been designed in a way that ensures that features that use JavaScript SDK will be used only after all the required components have been loaded.

All operations that use JavaScript SDK should be performed through the SE.ready() function, i.e. according to the following scheme:

SE.ready(function() {
    // Sphere Engine JavaScript SDK usage
});

Using the SE.ready() function ensures that the library has been loaded, the SE object has been initialized and no errors will occur while scripts are being executed.

Access to the widget

We gain access to the widget by using the SE.widget() function, which based on the given widget ID (previously defined by the data-id attribute) returns the object of SEWidget class used to manage the widget. For example:

SE.ready(function() {
    var widget = SE.widget("widget");
    // variable "widget" is ready to use
});

The SEWidget class provides the following options for managing the widget:

  • creating a widget,
  • dedicated events allowing for handling actions performed by the widget.

Methods

The global SE object provides public methods for managing widgets.

SE.widget()

An object of class SEWidget, which represent the widget, is created.

Parameters

Name Type Description
id string widget's identifier placed in the HTML document (data-id attribute)

Returned value

The object of class SEWidget is returned.

Example

Press the button to load widget
Load widget
<div data-id="example-widget" data-widget="{widget_hash}"></div>
<script>
    SE.ready(function() {
        $('#btn-load-widget').on('click', function(e) {
            e.preventDefault();
            var SEWidget = SE.widget("example-widget");
        });
    });
</script><a href="#" id="btn-load-widget">Load widget</a>

SEWidget.events.subscribe()

Assigns a function to the event (see list of events).

Parameters

Name Type Description
event string the name of the event to assign a function to
callback function function to be called on an event

Example

<div data-id="example-widget" data-widget="{widget_hash}"></div>
<script>
SE.ready(function() {

    var callback = function(data) {
        // Your code goes here
    };

    var SEWidget = SE.widget("example-widget");
    SEWidget.events.subscribe('{name_of_the_event}', callback);

});
</script>

SEWidget.events.unsubscribe()

Removes a function from the list of functions assigned to the event (see list of events).

Parameters

Name Type Description
event string the name of the event to remove the function from
callback function function to be removed from the list of functions assigned to the event

Example

<div data-id="example-widget" data-widget="{widget_hash}"></div>
<script>
SE.ready(function() {

    var callback = function(data) {
        // Your code goes here
    };

    var SEWidget = SE.widget("example-widget");
    SEWidget.events.unsubscribe('{name_of_the_event}', callback);

});
</script>

Events

An object of the SEWidget class has a collection of dedicated events called at specific moments of the widget's operation.

beforeSendSubmission

The event is invoked before sending a submission to the Sphere Engine system (i.e. after pressing the Submit button but before actually sending the submission).

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
submissionSource string the source code of the program sent in the submission
submissionLanguage string the programming language used to write the program

The value returned by the callback function

The value returned by the callback function determines whether the submission will be sent to the Sphere Engine system:

  • true - the submission is to be sent,
  • false - the submission will be stopped from being sent.

Example

Data:
Submit submission to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>
<script>
    SE.ready(function() {

        var beforeSendSubmission = function(data) {

            $('#result').html('')
                .append("> data.submissionLanguage: " + data.submissionLanguage + "<br><br>")
                .append("> data.submissionSource: <br>" + data.submissionSource + "<br><br>");

            return true;
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('beforeSendSubmission', beforeSendSubmission);
    });
</script>

Data:
<pre id="result" style="height: 300px;">Submit submission to see the result</pre>

afterSendSubmission

The event is invoked immediately after the submission has been sent to the Sphere Engine system.

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
submissionId integer submission identifier in the Sphere Engine system
submissionSource string the source code of the program sent in the submission
submissionLanguage string the programming language used to write the program

Example

Data:
Submit submission to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>
<script>
    SE.ready(function() {

        var afterSendSubmission = function(data) {

            $('#result').html('')
                .append("> data.submissionId: " + data.submissionId + "<br><br>");
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('afterSendSubmission', afterSendSubmission);
    });
</script>

Data:
<pre id="result" style="height: 100px;">Submit submission to see the result</pre>

checkStatus

The event is invoked regularly while checking the status of the submission during execution.

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
submissionId integer submission identifier in the Sphere Engine system
submissionTime integer execution time in seconds (available after execution)
submissionMemory integer memory consumption in kilobytes (available after execution)
statusNumber integer numeric value of the submission's status (see: list of statuses)
statusDescription string description of the submission's status

Example

Data:
Submit submission to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>

<script>
    SE.ready(function() {

        var checkStatus = function(data) {
            $('#result')
                .append("<br><br>")
                .append("> submissionId: " + data.submissionId + "<br>")
                .append("> statusNumber: " + data.statusNumber + "<br>")
                .append("> statusDescription: " + data.statusDescription + "<br>")
                .append("> submissionMemory: " + data.submissionMemory + "<br>")
                .append("> submissionTime: " + data.submissionTime + "<br>");

            $('#result').scrollTop($('#result').prop("scrollHeight"));
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('checkStatus', checkStatus);
    });
</script>

Data:
<pre id="result" style="height: 300px;">Submit submission to see the result</pre>

timeOver

The event is invoked after the time set for solving the task has expired.

Example

Data:
Start the session and wait 1 minute
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>

<script>
    SE.ready(function() {
        var timeOver = function() {
            $('#result').html('the session is over');
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('timeOver', timeOver);
    });
</script>

Data:
<pre id="result" style="height: 50px;">Start the session and wait 1 minute</pre>

welcomeUser

The event is invoked after sending the username and email address entered on the welcome page (available only for the configuration of the widget with the welcome page enabled).

Data sent to the callback function

An object with the following fields is sent to the callback function:

Name Type Description
userName string username provided on the welcome page
userEmail string email address provided on the welcome page

Example

Data:
Complete the form to see the result
<div data-id="example-widget" class="se-widget" data-widget="{widget_hash}"></div>

<script>
    SE.ready(function() {
        var welcomeUser = function(data) {
            $('#result').html('')
                .append("> userName: " + data.userName + "<br><br>")
                .append("> userEmail: " + data.userEmail + "<br><br>");
        };

        var SEWidget = SE.widget("example-widget");
        SEWidget.events.subscribe('welcomeUser', welcomeUser);
    });
</script>

Data:
<pre id="result" style="height: 100px;">Complete the form to see the result</pre>

Receiving the result of the submission

Sphere Engine Problems Widget allows you to configure the mode in which results are sent directly to the client's system.

The Callback URL field can be found in Menu > Problems > Widgets > WIDGET NAME > Widget settings. It can be used to specify the callback address. Sphere Engine uses this address to provide information about user submissions selected for assessment. Depending on the widget's settings, choosing a submission for assessment is automatic or manual - in both cases the information about the assessment is sent to the callback address.

Request specification

Sphere Engine sends a POST request to the callback address, providing information about the submission and the user in the body of the request in JSON format (i.e. the Content-type header with the value application/json is specified in the request).

The structure of the request's body is as follows:

{
    message:
    {
        id: <messageId>,
        name: <messageName>
    },
    body: {
        user: {
            uuid: <userUUID>,
            userId: <userCustomId>,
            userDefinedName: <userUserDefinedName>,
            userDefinedEmail: <userUserDefinedEmail>
        }
        context: {
            hash: <widgetHash>,
            name: <widgetName>
    }
        grade: {
            scoreInPoints: <scoreInPoints>,
            scorePercentage: <scorePercentage>
        }
    }
}

Parameters appearing in the message section:

  • messageId - unique message identifier,
  • messageName - message type (currently only "sendGrade" is available).

Parameters appearing in the body.user section:

  • userUUID - unique user ID in the Sphere Engine system,
  • userCustomId - user ID defined in the client's system,
  • userUserDefinedName - user name entered in the welcome form,
  • userUserDefinedEmail - email address entered in the welcome form.

Parameters appearing in the body.context section:

  • widgetHash - a unique widget identifier in the Sphere Engine system,
  • widgetName - name of the widget.

Parameters appearing in the body.grade section:

  • scoreInPoints - number of points obtained for the submission,
  • scorePercentage - percentage ratio in relation to the best submission.

Response structure

After sending the result of the submission, the Sphere Engine system waits for confirmation of receipt of the message from the client's system.

The expected response should have HTTP status code 200 and be in JSON format (including the HTTP header Content-type: application/json) with the following structure:

{
    status: <receptionStatus>
}

The receptionStatus parameter can take one of two values:

  • success - in the case of successful receipt and processing of information about the result of the submission,
  • failure - in case of a failure.

In the case of a negative response from the client's system (i.e. an HTTP response with a status code other than 200 or a value of the receptionStatus parameter other than "success"), the Sphere Engine system will attempt to resend the information about the result of the submission in accordance with the schedule.

Notification schedule

In the case of a negative response from the client's system, the Sphere Engine system will attempt to resend the information about the result of the submission in accordance with the schedule.

Re-attempts to send information are made with a delay dependent on the number of the attempt in accordance with the table below:

Attempt no. Delay
1 - 10 1 minute
11 - 20 5 minutes
21 - 30 10 minutes
31 - 40 30 minutes
41 - 50 1 hour
51 - 60 6 hours

Carrying out further attempts with increasing delay time secures the client's system against data loss during unavailability (e.g. during a failure). After 60 unsuccessful attempts, the Sphere Engine system stops sending information about the result of the submission.

Note: All 60 attempts take more than 75 hours to complete. In most cases, this is sufficient time to perform repairs in the event of the client's system failure. If the system is unavailable for a longer period, you can contact technical support at support@sphere-engine.com to establish when the sending of information shall resume.

Message signature

The return request sent by the Sphere Engine system is signed using the signature algorithm (discussed in the section devoted to security). Validation of the signature in the client's system is not mandatory, although it is recommended for security reasons.

Signature parameters

Signature verification requires taking into account the signature, widget parameters and additional signature parameters used to generate the signature:

  • hash - unique widget ID in the Sphere Engine system. It can be found in the Unique hash field inMenu > Problems > Widgets > WIDGET NAME > Widget settings,
  • se_body_hash - the effect of the SHA-1 hash function on the body of the return request,
  • se_nonce - unique message identifier,
  • se_secret - secret key that prevents signature falsification,
  • se_signature - signature.

Important: The se_secret parameter is the basis for the security of the signature and should remain a secret. Do not send it in messages between systems.

Location of the parameters required to verify the signature:

Parameter name Location or generation method
hash included in the body of the return request (key: body.context.hash)
se_body_hash value of the SHA-1 hash function for the body of the return request
se_nonce the value of the HTTP header named SphereEngineNonce
se_secret available in the Shared secret field in Menu > Problems > Widgets > WIDGET NAME > Widget settings
se_signature the value of the HTTP header named SphereEngineSignature
Signature verification algorithm

The presented algorithm uses the signature generation algorithm discussed earlier.

verify_signature(headers, request_body) {
    var signature = headers["SphereEngineSignature"];

    var params = [
        "hash": request_body.body.context.hash,
        "se_body_hash": sha1(request_body),
        "se_nonce": headers["SphereEngineNonce"],
        "se_secret": CONFIG["SEPW_SHARED_SECRET"]
    ]

    return signature == generate_signature(params);
}

Message ID (nonce)
With each message, Sphere Engine transmits its unique identifier (se_nonce parameter) to secure the system against repeat processing of the same message.

It is up to the client whether to use this parameter. Checking the uniqueness of the se_nonce parameter is optional but it is recommended for security reasons.