Developer Guide

Start here

Getting started

First of all, thank you for your interest in contributing to Kolibri! The project was founded by volunteers dedicated to helping make educational materials more accessible to those in need, and every contribution makes a difference. The instructions below should get you up and running the code in no time!

Setting up Kolibri for development

Most of the steps below require entering commands into your Terminal (Linux, Mac) or command prompt (cmd.exe on Windows) that you will learn how to use and become more comfortable with.

Tip

In case you run into any problems during these steps, searching online is usually the fastest way out: whatever error you are seeing, chances are good that somebody already had it in the past and posted a solution somewhere… ;)

Git & GitHub
  1. Install and set-up Git on your computer. Try this tutorial if you need more practice with Git!
  2. Sign up and configure your GitHub account if you don’t have one already.
  3. Fork the main Kolibri repository. This will make it easier to submit pull requests. Read more details about forking from GitHub.
Install Environment Dependencies

  1. Install Python if you are on Windows, on Linux and OSX Python is preinstalled (recommended versions 2.7+ or 3.4+).

  2. Install pip package installer.

  3. Install Node (version 6 is required).

  4. Install Yarn according the instructions specific for your OS.

    Note

    • On Ubuntu install Node.js via nvm to avoid build issues.
    • On a Mac, you may want to consider using the Homebrew package manager.

Ready for the fun part in the Terminal? Here we go!

Checking out the code

  1. Make sure you registered your SSH keys on GitHub.

  2. Clone your Kolibri fork to your local computer. In the following commands replace $USERNAME with your own GitHub username:

    # using SSH
git clone git@github.com:$USERNAME/kolibri.git
# using HTTPS
git clone https://github.com/$USERNAME/kolibri.git

  3. Enable syncing your local repository with upstream, which refers to the Kolibri source from where you cloned your fork. That way you can keep it updated with the changes from the rest of Kolibri team contributors:

cd kolibri  # Change into the newly cloned directory
git remote add upstream git@github.com:learningequality/kolibri.git  # Add the upstream
git fetch upstream # Check if there are changes upstream
git checkout develop

Warning

develop is the active development branch - do not target the master branch.

Virtual environment

It is best practice to use Python virtual environment to isolate the dependencies of your Python projects from each other. This also allows you to avoid using sudo with pip, which is not recommended.

You can learn more about using virtualenv, or follow these basic instructions:

Initial setup, performed once:

$ sudo pip install virtualenv  # install virtualenv globally
$ mkdir ~/.venvs               # create a common directory for multiple virtual environments
$ virtualenv ~/.venvs/kolibri  # create a new virtualenv for Kolibri dependencies

Note

We create the virtualenv outside of the Kolibri project folder. You can choose another location than ~/.venvs/kolibri if desired.

To activate the virtualenv in a standard Bash shell:

$ source ~/.venvs/kolibri/bin/activate  # activate the venv

Now, any commands run with pip will target your virtualenv rather than the global Python installation.

To deactivate the virtualenv, run the command below. Note, you’ll want to leave it activated for the remainder of project setup!

$ deactivate

Tip

  • Users of Windows and other shells such as Fish should read the guide for instructions on activating.
  • If you set the PIP_REQUIRE_VIRTUALENV environment variable to true, pip will only install packages when a virtualenv is active. This can help prevent mistakes.
  • Bash users might also consider using virtualenvwrapper, which simplifies the process somewhat.
Install Project Dependencies

Note

Make sure your virtualenv is active!

To install Kolibri project-specific dependencies make sure you’re in the kolibri directory and run:

# Python requirements
(kolibri)$ pip install -r requirements.txt
(kolibri)$ pip install -r requirements/dev.txt

# Kolibri Python package in 'editable' mode, so your installation points to your git checkout:
(kolibri)$ pip install -e .

# Javascript dependencies
(kolibri)$ yarn install

Tip

  • We’ve adopted this concatenated version with added cleanup: make clean && pip install -r requirements.txt --upgrade && pip install -e . && yarn install.
  • In case you get webpack compilation error with Node modules build failures, add the flag --force at the end, to ensure binaries get installed.

Running Kolibri server

Development server

To start up the development server and build the client-side dependencies, use the following command:

(kolibri)$ yarn run devserver

If this does not work, you can run the command that this is invoking:

(kolibri)$ kolibri --debug manage devserver --webpack --lint --settings=kolibri.deployment.default.settings.dev

Wait for the build process to complete. This takes a while the first time, will complete faster as you make edits and the assets are automatically re-built.

Now you should be able to access the server at http://127.0.0.1:8000/.

Tip

If you need to make the development server available through the LAN, you must leave out the --webpack flag, and use the following command:

(kolibri)$ yarn run build
(kolibri)$ kolibri --debug manage devserver -- 0.0.0.0:8000

Now you can simply use your server’s IP from another device in the local network through the port 8000, for example http://192.168.1.38:8000/.

Tip

If get an error similar to Node Sass could not find a binding for your current environment try running:

(kolibri)$ npm rebuild node-sass

More advanced examples of the devserver command:

# runs the dev server, rebuild client assets when files change, and use developer settings
kolibri --debug manage devserver --webpack --settings=kolibri.deployment.default.settings.dev

# runs the dev server and re-run client-side tests when files changes
kolibri --debug manage devserver --karma

# runs all of the above
kolibri --debug manage devserver --webpack --karma --settings=kolibri.deployment.default.settings.dev
Running the Production Server

In production, content is served through CherryPy. Static assets must be pre-built:

yarn run build
kolibri start

Now you should be able to access the server at http://127.0.0.1:8080/.

Development

Linting

To improve build times, and facilitate rapid development, Javascript linting is turned off by default when you run the dev server. However, all frontend assets that are bundled will be linted by our Travis CI builds. It is a good idea, therefore, to test your linting before submitting code for PR. To run the devserver in this mode you can run the following command.

kolibri --debug manage devserver --webpack --lint
Code Testing

Kolibri comes with a Python test suite based on py.test. To run tests in your current environment:

pytest  # alternatively, "make test" does the same

You can also use tox to setup a clean and disposable environment:

tox -e py3.4  # Runs tests with Python 3.4

To run Python tests for all environments, lint and documentation tests, use simply tox. This simulates what our CI also does.

To run Python linting tests (pep8 and static code analysis), use tox -e lint or make lint.

Note that tox reuses its environment when it is run again. If you add anything to the requirements, you will want to either delete the .tox directory, or run tox with the -r argument to recreate the environment.

We strive for 100% code coverage in Kolibri. When you open a Pull Request, code coverage (and your impact on coverage) will be reported. To test code coverage locally, so that you can work to improve it, you can run the following:

tox -e py3.4
coverage html

Then, open the generated ./htmlcov/index.html file in your browser.

Kolibri comes with a Javascript test suite based on mocha. To run all tests:

yarn test

This includes tests of the bundling functions that are used in creating front end assets. To do continuous unit testing for code, and jshint running:

yarn run test-karma:watch

Alternatively, this can be run as a subprocess in the development server with the following flag:

kolibri --debug manage devserver --karma

You can also run tests through Django’s test management command, accessed through the kolibri command:

kolibri manage test

To run specific tests only, you can add --, followed by a label (consisting of the import path to the test(s) you want to run, possibly ending in some subset of a filename, classname, and method name). For example, the following will run only one test, named test_admin_can_delete_membership in the MembershipPermissionsTestCase class in kolibri/auth/test/test_permissions.py:

kolibri manage test -- kolibri.auth.test.test_permissions.MembershipPermissionsTestCase.test_admin_can_delete_membership

To run a subset of tests, you can also run

py.test test/test_kolibri.py
Updating Documentation

First, install some additional dependencies related to building documentation output:

pip install -r requirements/docs.txt
pip install -r requirements/build.txt

To make changes to documentation, edit the rst files in the kolibri/docs directory and then run:

make docs

You can also run the auto-build for faster editing from the docs directory:

cd docs
sphinx-autobuild --port 8888 . _build
Manual Testing

All changes should be thoroughly tested and vetted before being merged in. Our primary considerations are:

  • Performance
  • Accessibility
  • Compatibility
  • Localization
  • Consistency

For more information, see the next section on Manual testing & QA.

Submitting a Pull Request

The most common situation is working off of develop branch so we’ll take it as an example:

$ git checkout upstream/develop
$ git checkout -b name-of-your-bugfix-or-feature

After making changes to the code, commit and push them to a branch on your fork:

$ git add -A  # Add all changed and new files to the commit
$ git commit -m "Write here the commit message"
$ git push origin name-of-your-bugfix-or-feature

Go to Kolibri GitHub page, and if you are logged-in you will see the link to compare your branch and and create the new pull request. Please fill in all the applicable sections in the PR template and DELETE unecessary headings. Another member of the team will review your code, and either ask for updates on your part or merge your PR to Kolibri codebase. Until the PR is merged you can push new commits to your branch and add updates to it.

Learn more about our Release process

Tech stack overview

Kolibri is a web application built primarily using Python on the server-side and JavaScript on the client-side.

We use many run-time, development, and build-related technologies and tools, as outlined below.

Server

The server is a Django 1.9 application, and contains only pure-Python (2.7+) libraries dependencies at run-time.

The server is responsible for:

  • Interfacing with the database (PostgreSQL) containing user, content, and language pack data
  • Authentication and permission middleware
  • Routing and handling of API calls, using the Django REST Framework
  • Basic top-level URL routing between high-level sections of the application
  • Serving basic HTML wrappers for the UI with data bootstrapped into the page
  • Serving additional client assets such as fonts and images

TODO - how does Morango fit into this picture? Logging?

Client

The front-end user interface is built using HTML, the Stylus CSS-preprocessing language, and the ES2015 preset features of ES6 JavaScript.

The frontend targets IE9 and up, with an emphasis on tablet-size screens. We strive to use accessible, semantic HTML with support for screen readers, keyboard interaction, and right-to-left language support.

The client is responsible for:

  • Compositing and rendering the UI using Vue.js components to build nested views
  • Managing client-side state using Vuex
  • Interacting with the server through HTTP requests

Additionally, loglevel is used for logging and normalize.css is included for browser style normalization.

Internationalization

We leverage the ICU Message syntax for formatting all user-facing text.

See Internationalization for more information.

Developer Docs

Documentation for Kolibri developers are formatted using reStructuredText and the output is generated using Sphinx. Most of the content is in the /docs directory, but some content is also extracted from Python source code and from files in the root directory. We use Read the Docs to host a public version of our documentation.

Additionally, information about the design and implementation of Kolibri might be found on Google Drive, Trello, Slack, InVision, mailing lists, office whiteboards, and lurking in the fragmented collective consciousness of our contributors.

Build Infrastructure

Client-side Resources

We use a combination of both Node.js and Python scripts to transform our source code as-written to the code that is run in a browser. This process involves webpack, plus a number of both custom and third-party extensions.

Preparation of client-side resources involves:

  • ES6 to ES5
  • Transforming Vue.js component files (*.vue) into JS and CSS
  • Stylus to CSS
  • Auto-prefixing CSS
  • Bundling multiple JS dependencies into single files
  • Minifying and compressing code
  • Bundle resources such as fonts and images
  • Generating source maps
  • Providing mechanisms for decoupled “Kolibri plugins” to interact with each other and asynchronously load dependencies
  • Linting to enforce code styles
Server Setup

The standard Django manage.py commands are used under-the-hood for database migration and user set-up.

TODO: is this accurate?

Installers and Packages

TODO: introduce stack (sdist, PyPi, Debian, Windows, etc)

see Distribution build pipeline

Continuous Integration

TODO: introduce stack (GitHub, CodeCov, Travis, buildkite, commit hooks)

Tests and Linting

We use a number of mechanisms to help encourage code quality and consistency. These checks enforce a subset of our Front-end code conventions.

  • pre-commit is run locally on git commit and enforces some Python conventions
  • We use EditorConfig to help developers set their editor preferences
  • flake8 is also used to enforce Python conventions
  • tox is used to run our test suites under a range of Python and Node environment versions
  • sphinx-build -b linkcheck checks the validity of documentation links
  • pytest runs our Python unit tests. We also leverage the Django test framework.
  • In addition to building client assets, webpack runs linters on client-side code: ESLint for ES6 JavaScript, Stylint for Stylus, and HTMLHint for HTML and Vue.js components.
  • Client-side code is tested using a stack of tools including Karma, Mocha, PhantomJS, Sinon, and rewire. TODO: Explain what each of these do
  • codecov reports on the test coverage for Python and Node.js code. TODO - also client-side?

Helper Scripts

TODO: introduce stack (kolibri command, setup.py, makefiles, yarn commands, sphinx auto-build, etc)

References

Manual testing & QA

Accessibility (A11y) Testing

Inclusive design benefits all users, and we strive to make Kolibri accessible for all. Testing for accessibility can be challenging, but there are a few features you should check for before submitting your PR:

  • Working keyboard navigation - everything that user can do with mouse or by touch must also work with the keyboard alone.
  • Sufficient color contrast between foreground text/elements and the background.
  • Meaningful text alternative for all non-decorative images, or an empty ALT attribute in case of decorative ones.
  • Meaningful labels on ALL form or button elements.
  • Page has one main heading (H1) and consecutive lower heading levels.

Here are a few tools that we use in testing for accessibility:

There is a much longer list on our Kolibri Accessibility Tools Wiki page if you want to go deeper, but these four should be enough to help you avoid the most important accessibility pitfalls.

Cross-browser and OS Testing

It’s vital to ensure that our app works across a wide range of browsers and operating systems, particularly older versions of Windows and Android that are common on old and cheap devices.

In particular, we want to ensure that Kolibri runs on major browsers that match any of the following criteria:

  • within the last two versions
  • IE 9+ on Windows XP and up
  • has at least 1% of global usage stats

Here are some useful options, in order of simplicity:

BrowserStack

BrowserStack is an incredibly useful tool for cross-browser and OS testing. In particular, it’s easy to install plugin which forwards localhost to a VM running on their servers, which in turn is displayed in your browser.

Amazon Workspaces

In some situations, simply having a browser is not enough. For example, a developer may need to test Windows-specific backend or installer code from another OS. In many situations, a virtual machine is appropriate - however these can be slow to download and run.

Amazon’s AWS Workspaces provides a faster alternative. They run Windows VMs in their cloud, and developers can RDP in.

Local Virtual Machines

Workspaces is very useful, but it has limitations: only a small range of OSes are available, and connectivity and provisioning are required.

An alternative is to run the guest operating system inside a virtual machine using e.g. VirtualBox. This also gives more developer flexibility, including e.g. shared directories between the guest and host systems. This tutorial shows how to test Kolibri in a VM.

Hardware

There are some situations where actual hardware is necessary to test the application. This is particularly true when virtualization might prohibit or impede testing features, such as lower-level driver interactions.

Responsiveness to Varying Screen Sizes

We want to ensure that the app looks and behaves reasonably across a wide range of typical screen sizes, from small tablets to large, HD monitors. It is highly recommended to constantly be testing functionality at a range of sizes.

Chrome and Firefox’s Developer Tools both have some excellent functionality to simulate arbitrary screen resolutions.

Slow Network Connection Speeds

It’s important to simulate end-users network conditions. This will help identify real-world performance issues that may not be apparent on local development machines.

Chrome’s Developer Tools have functionality to simulate a variety of network connections, including Edge, 3G, and even offline. An app can be loaded into multiple tabs, each with its own custom network connectivity profile. This will not affect traffic to other tabs.

Within the Chrome Dev Tools, navigate to the Network panel. Select a connection from the drop-down to apply network throttling and latency manipulation. When a Throttle is enabled the panel indicator will show a warning icon. This is to remind you that throttling is enabled when you are in other panels.

For Kolibri, our target audience’s network condition can be mimicked by setting connectivity to Regular 3G (100ms, 750kb/s, 250 kb/s).

Performance Testing with Django Debug Panel

We have built in support for Django Debug Panel (a Chrome extension that allows tracking of AJAX requests to Django).

To use this, ensure that you have development dependencies installed, and install the Django Debug Panel Chrome Extension. You can then run the development or production servers with the following environment variable set:

DJANGO_SETTINGS_MODULE=kolibri.deployment.default.settings.debug_panel

This will activate the debug panel, and will display in the Dev tools panel of Chrome. This panel will track all page loads and API requests. However, all data bootstrapping into the template will be disabled, as our data bootstrapping prevents the page load request from being profiled, and also does not profile the bootstrapped API requests.

Generating User Data

For manual testing, it is sometimes helpful to have generated user data, particularly for Coach and Admin facing functionality.

In order to do this, a management command is available:

kolibri manage generateuserdata

This will generate user data for the each currently existing channel on the system. Use the –help flag for options.

Release process

Branches and tags

  • The master branch always has the latest stable code
  • The develop branch is our current development branch
  • Branches named like release-v1.2.x (for example) track all releases of the 1.2 release line. This may include multiple patch releases (like v1.2.0, v1.2.1, etc)
  • Tags named like like v1.2.0-beta1 and v1.2.0 label specific releases

Note

At a high level, we follow the ‘Gitflow’ model. Some helpful references: Original reference, Atlassian

If a change needs to be introduced to an old release, target the oldest release branch that you want a bug fix introduced in. Then that will be merged into all later releases, including develop.

When we get close to releasing a new stable version/release of Kolibri, we generally branch develop into something like release-v0.1.x and tag it as a new beta. If you’re working on an issue targetted with that milestone, then you should target changes to that branch. Changes to those branches will later be pulled into develop again.

If you’re not sure which branch to target, ask the dev team!

Process

Update the Changelog

Update the Release Notes as necessary. In general we should try to keep the changelog up-to-date as PRs are merged in; however in practice the changelog usually needs to be cleaned up, fleshed out, and clarified.

Our changelogs should list:

  • significant new features that were added
  • significant categories of bug fixes or user-facing improvements
  • significant behind-the-scenes technical improvements

Keep entries concise and consistent with the established writing style. The changelog should not include an entry for every PR or every issue closed. Reading the changelog should give a quick, high-level, semi-technical summary of what has changed.

Note that for older patch releases, the change should only be mentioned once: it is implied that fixes in older releases are propagated forward.

Create a release branch

If this is a new major or minor release, you need to make a new branch as described above.

Pin installer versions

On Kolibri’s develop branch, we sometimes allow the installers to track the latest development versions on github. Before releasing Kolibri, we need to pin the Buildkite configuration to a tagged version of each installer.

Update any translation files

If string interface text has changed, or more complete translations are available, translation files should be updated. This is currently done by running the make downloadmessages command. Following this, the specific files that have been updated with approved translations will need to be added to the repository.

Caveats:

  • The crowdin utility that this command invokes requires java, so you may need to run them in an ubuntu VM
  • You might need to manually install the crowdin debian package if the jar isn’t working for you
  • The command might not be compatible with non-bash shells
  • You might be better off composing the crowdin commands manually, especially if your checked out branch is not a release branch
  • By default Crowdin will download all translations, not just approved ones, and will often download untranslated strings also. Do not just add all the files that are downloaded when make downloadmessages is run, as this will lead to untranslated and poor quality strings being included.

If you need to add a new interface language to Kolibri, please see new_language for details.

Finally, strings for any external Kolibri plugins (like kolibri-exercise-perseus-renderer) should also have been updated, a new release made, and the version updated in Kolibri. See the README of that repository for details.

Squash migrations

When possible, we like to utilize the Django migration squashing to simplify the migration path for new users (while simultaneously maintaining the migration path for old users). So far this has not been done, due to the existence of data migrations in our migration history. Once we have upgraded to Django 1.11, we will be able to mark these data migrations as elidable, and we will be able to better squash our history.

Ensure bugfixes from internal depencies have propagated

Some issues in Kolibri arise due to our integration of internally produced, but external to Kolibri, packages, such as kolibri-exercise-perseus-renderer, iceqube, and morango. If any of these kinds of dependencies have been updated to fix issues for this milestone, then the dependency version should have been updated.

Edit the VERSION file

Current practice is to bump kolibri.VERSION before tagging a release. You are allowed to have a newer version in kolibri.VERSION, but you are not allowed to add the tag before actually bumping kolibri.VERSION.

Current practice is to bump kolibri.VERSION before tagging a release. You are allowed to have a newer version in kolibri.VERSION, but you are not allowed to add the tag before actually bumping kolibri.VERSION.

Select a release series number and initial version number:

$ SERIES=0.1.x
$ VER=0.1.0a

The form is:

       0.1.x
      /  |  \
     /   |   \
    /    |    \
major  minor   patch

Set the version in the release branch:

$ # edit VERSION in kolibri/__init__.py
$ git add kolibri/__init__.py
$ git commit -m "Bump version to $VER"

Set the version number in the develop branch if necessary.

Create a pull request on Github to get sign off for the release.

Checklist for sign off:

  • [ ] Translation files have been updated
  • [ ] Migrations have been squashed where possible
  • [ ] Changelog has been updated
  • [ ] LE Dependencies properly updated
  • [ ] Tested Debian Installer
  • [ ] Tested Windows Installer
  • [ ] Tested PEX File
Tag the release

We always add git tags to a commit that makes it to a final or pre release. A tag is prefixed v and follows the Semver convention, for instance v1.2.3-alpha1.

Tag the release using github’s Releases feature.

Once a stable release is tagged, delete pre-releases (not the tags themselves) from github.

Copy the entries from the changelog into Github’s “Release notes”.

Warning

Always add tags in release branches. Otherwise, the tag chronology will break. Do not add tags in feature branches or in the master branch. You can add tags for pre-releases in develop, for releases that don’t yet have a release branch.

Warning

Tagging is known to break after rebasing, so in case you rebase a branch after tagging it, delete the tag and add it again. Basically, git describe --tags detects the closest tag, but after a rebase, its concept of distance is misguided.

Release to PyPI

Select the version number and checkout the exact git tag:

$ VER=0.1.0
$ git checkout v$VER

Release with PyPI using the make command:

$ make release

Declare victory.

Post-release TODO

Most of these TODOs are targeted towards more public distribution of Kolibri, and as such have not been widely implemented in the past. Once Kolibri is publicly released, these will be increasingly important to support our community.

  • Release on PyPI
  • Update any redirects on learningequality.org for the latest release.
  • Announce release on dev list and newsletter if appropriate.
  • Close, if fixed, or change milestone of any issues on this release milestone.
  • Close this milestone.
  • For issues on this milestone that have been reported by the community, respond on the issues or other channels, notifying of the release that fixes this issues.

More on version numbers

Note

The content below is pulled from the docstring of the kolibri.utils.version module.

We follow semantic versioning 2.0.0 according to semver.org but for Python distributions and in the internal string representation in Python, you will find a PEP-440 flavor.

  • 1.1.0 (Semver) = 1.1.0 (PEP-440).
  • 1.0.0-alpha1 (Semver) = 1.0.0a1 (PEP-440).

Here’s how version numbers are generated:

  • kolibri.__version__ is automatically set, runtime environments use it to decide the version of Kolibri as a string. This is especially something that PyPi and setuptools use.
  • kolibri.VERSION is a tuple containing version information, it’s set in kolibri/__init__.py is automatically suffixed in pre-releases by a number of rules defined below. For a final release (not a pre-release), it will be used exactly as it appears.
  • kolibri/VERSION is a file containing the exact version of Kolibri for a distributed environment (pre-releases only!)
  • git describe --tags is a command run to fetch tag information from a git checkout with the Kolibri code. The information is used to validate the major components of kolibri.VERSION and to suffix the final version of prereleases. This information is stored permanently in kolibri/VERSION before shipping a pre-release by calling make writeversion during make dist etc.

Confused? Here’s a table:

Release type kolibri.VERSION kolibri/VERSION Git data Examples
Final Canonical, only information used N/A N/A 0.1.0, 0.2.2, 0.2.post1
dev release (alpha0) (1, 2, 3, ‘alpha’, 0), 0th alpha = a dev release! Never used as a canonical Fallback timestamp of latest commit + hash 0.4.0.dev020170605181124-f1234567
alpha1+ (1, 2, 3, ‘alpha’, 1) Fallback git describe --tags Clean head: 1.2.3a1, Changes since tag: 1.2.3a1.dev123-f1234567
beta1+ (1, 2, 3, ‘alpha’, 1) Fallback git describe --tags Clean head: 1.2.3b1, Changes since tag: 1.2.3b1.dev123-f1234567
rc1+ (release candidate) (1, 2, 3, ‘alpha’, 1) Fallback git describe --tags Clean head: 1.2.3rc1, Changes since tag: 1.2.3rc1.dev123-f1234567
beta0, rc0, post0, x.y.0 Not recommended, but if you use it, your release transforms into a X.Y.0b0.dev{suffix} release, which in most cases should be assigned to the preceding release type. Fallback timestamp of latest commit + hash 0.4.0b0.dev020170605181124-f1234567

Fallback: kolibri/VERSION is auto-generated with make writeversion during the build process. The file is read as a fallback when there’s no git data available in a pre-release (which is the case in an installed environment).

Release order example 1.2.3 release:

  • VERSION = (1, 2, 3, 'alpha', 0) throughout the development phase, this results in a lot of 1.2.3.dev0YYYYMMDDHHMMSS-1234abcd with no need for git tags.
  • VERSION = (1, 2, 3, 'alpha', 1) for the first alpha release. When it’s tagged and released,

Warning

Do not import anything from the rest of Kolibri in this module, it’s crucial that it can be loaded without the settings/configuration/django stack.

Do not import this file in other package’s __init__, because installation with setup.py should not depend on other packages. In case you were to have a package foo that depended on kolibri, and kolibri is installed as a dependency while foo is installing, then foo won’t be able to access kolibri before after setuptools has completed installation of everything.

Internationalization

As a platform intended for use around the world, Kolibri has a strong mandate for translation and internationalization. As such, it has been designed with technologies to enable this built in.

Backend Translation

For any strings in Django, we are using the standard Django i18n machinery (gettext and associated functions) to provide translations. See the Django i18n documentation for more information.

Frontend Translation

For any strings in the frontend, we are using Vue-Intl an in house port of React-intl.

Within Kolibri, messages are defined on the body of the Vue component:

- ``$trs``, an object of the form::

  {
    msgId: 'Message text',
  }

- ``name``, we use the Vue component name to namespace the messages.

The name and all ``msgId``s should be in camelCase.

User visible strings should be rendered directly in the template with {{ $tr('msgId') }}. These strings are collected during the build process, and bundled into exported JSON files. These files are then uploaded to Crowdin for translation.

An example Vue component would then look like this:

<template>

  <div>
    <p>{{ $tr('exampleMessage') }}</p>
  </div>

</template>


<script>

  module.exports = {

    name: 'example',
    $trs: {
      exampleMessage: 'This message is just an example',
    },
  };

</script>


<style lang="stylus" scoped></style>

In order to translate strings outside of the scope of Vue components, i.e. in Javascript source files, the name space and messages object still need to be defined, as shown in this example:

import { createTranslator } from 'kolibri.utils.i18n';

const name = 'exampleTitles';

const messages = {
  msgIdForThisMessage: 'This is a message',
};

const translator = createTranslator(name, messages);

console.log(translator.$tr('msgIdForThisMessage'));

In this way, messages are namespaced, and then available off the $tr method of the translator object returned from the createTranslator function.

These messages will then be discovered for any registered plugins and loaded into the page if that language is set as the Django language. All language setting for the Frontend is based off the current Django language for the request.

Adding a new language

In order to add a new supported language to Kolibri, the appropriate language tuple must be added to the LANGUAGES variable in kolibri/deployment/default/settings/base.py. In addition, the appropriate Intl polyfill file must be added to kolibri/core/assets/src/utils/import-intl-locale.js.

Front-end code conventions

Vue.js Components

Note that the top-level tags of Vue.js components are <template>, <script>, and <style>.

  • Whitespace
    • an indent is 2 spaces
    • two blank lines between top-level tags
    • one blank line of padding within a top-level tag
    • one level of indent for the contents of all top-level tags
  • Keep most child-components stateless. In practice, this means using props but not data.
  • Avoid using Vue.js’ camelCase-to-kebab-case mapping. Instead, use square brackets and strings to reference names.
  • Use scoped styles where ever possible
  • Name custom tags using kebab-case
  • Components are placed in the vue directory. The root component file is called vue/index.vue, and is mounted on a tag called <rootvue>.
  • Components are defined either as a file with a .vue extension (my-component.vue) or as a directory with an index.vue file (my-component/index.vue). Both forms can be used with require('my-component').
  • Put child components inside the directory of a parent component if they are only used by the parent. Otherwise, put shared child components in the vue director.
  • Any user visisble interface text should be rendered translatable, see Internationalization for details.

Stylus and CSS

  • clear out unused styles
  • avoid using classes as JS identifiers, and prefix with js- if necessary

HTML

attribute lists, semantic structure, accessibility…

Files and directories

.cache/…
Testing-related, and ignored by git. TODO - what does it contain?
.eggs/…
Packaging-related, and ignored by git. TODO - what does it contain?
.github/…
These are files used by GitHub to generate templates for things like new pull requests and issues.
.tox/…

Tox is a tool for testing software in a range of environments - for example using different versions of Python and Node.

This directory is ignored by git.

TODO - what does it contain?

dist-packages-cache
Packaging-related, and ignored by git. TODO - what does it contain?
dist-packages-temp
Packaging-related, and ignored by git. TODO - what does it contain?
docs/…
reStructuredText-based documentation, along with Sphinx-based build code
frontend_build/…
Code for integrating Kolibri’s plugin system with webpack instrumentation for bundling client-side dependencies.
karma_config/…
Configuration for Karma, our client-side unit test framework
kolibri/…
main code-base, a Django application
requirements/…
Python dependency files for PIP
test/…
helper files for running tests in Travic CI TODO - is this correct?
.editorconfig
general editor configuration file
.eslintrc.js
configuration file for ESLint, our client-side javascript linter
.gitignore
standard .gitignore file
.htmlhintrc
configuration for our HTML linter, HTMLHint
.pre-commit-config.yaml
configuration for our pre-commit hooks
.stylintrc
configuration for our Stylus linter, Stylint
.travis.yml
configuration for Travis
AUTHORS.rst, CHANGELOG.rst, CONTRIBUTING.rst
reStructuredText-formatted files. Also imported by the generated /docs
LICENSE
plain-text license files
Makefile
wrapper for some scripts, including building packages and docs
MANIFEST.in
list of non-python files to include in the Python package
package.json
javascript dependencies, helper scripts, and configuration
pytest.ini
configuration file for pytest
pytest_runner-2.7.1-py2.7.egg
?
README.rst
reStructuredText-formatted file readme
requirements.txt
Python PIP dependency requirements, simply redirects to requirements/base.txt
setup.cfg
?
setup.py
configuration for Python package related to setuptools
tox.ini
configuration for our Tox test environments

Architecture

Front-end architecture

Components

We leverage Vue.js components as the primary building blocks for our UI. For general UI development work, this is the most common tool a developer will use. It would be prudent to read through the Vue.js guide thoroughly.

Each component contains HTML with dynamic Vue.js directives, styling which is scoped to that component (written using Stylus), and logic which is also scoped to that component (all code, including that in Vue components should be written using Bublé compatible ES2015 JavaScript). Non-scoped styles can also be added, but these should be carefully namespaced.

Components allow us to define new custom tags that encapsulate a piece of self-contained, re-usable UI functionality. When composed together, they form a tree structure of parents and children. Each component has a well-defined interface used by its parent component, made up of input properties, events and content slots. Components should never reference their parent.

Read through Front-end code conventions for some important consistency tips on writing new components.

Layout of Frontend Code

Front-end code and assets are generally contained in one of two places: either in one of the plugin subdirectories (under kolibri/plugins) or in kolibri/core, which contains code shared across all plugins as described below.

Within these directories, there should be an assets directory with src and test under it. Most assets will go in src, and tests for the components will go in test.

For example:

kolibri/
  core/                       # core (shared) items
    assets/
      src/
        core-base.vue         # global base template, used by apps
        core-modal.vue        # example of another shared component
        core-global.styl      # globally defined styles, indluded in head
        core-theme.styl       # style variable values
        font-NotoSans.css     # embedded font
      test/
        ...                   # tests for core assets
  plugins/
    learn                     # learn plugin
      assets/
        src/
          vue/
            index.vue         # root view
            some-page.vue     # top-level client-side page
            another-page/     # top-level client-side page
              index.vue
              child.vue       # child component used only by parent
            shared.vue        # shared across this plugin
          app.js              # instantiate learn app on client-side
          router.js
          store.js
        test/
          app.js
    management/
      assets/
        src/
          vue/user-page.vue   # nested-view
          vue/index.vue       # root view
          app.js              # instantiate mgmt app on client-side
        test/
          app.js

In the example above, the vue/another-page/index.vue file in learn can use other assets in the same directory (such as child.vue), components in vue (such as shared.vue), and assets in core (such as variables in core-theme.styl). However it cannot use files in other plugin directories (such as management).

Note

For many development scenarios, only files in these directories need to be touched.

There is also a lot of logic and configuration relevant to front-end code loading, parsing, testing, and linting. This includes webpack, NPM, and integration with the plugin system. This is somewhat scattered, and includes logic in frontend_build/…, package.json, kolibri/core/webpack/…, and other locations. Much of this functionality is described in other sections of the docs (such as asset_loading), but it can take some time to understand how it all hangs together.

SVG Icons

SVGs can be inlined into Vue components using a special syntax:

<svg src="icon.svg"></svg>

Then, if there is a file called icon.svg in the same directory, that file will be inserted directly into the outputted HTML. This allows aspects of the icon (e.g. fill) to be styled using CSS.

Attributes (such as vue directives like v-if and SVG attributes like viewbox) can also be added to the svg tag.

Single-page Apps

The Kolibri front-end is made of a few high-level “app” plugins, which are single-page JS applications (conventionally app.js) with their own base URL and a single root Vue.js component. Examples of apps are ‘Learn’ and ‘User Management’, as shown in the example above. Apps are independent of each other, and can only reference components and styles from within themselves and from core.

Each app is implemented as a Kolibri plugin and is defined in a subdirectory of kolibri/plugins.

On the Server-side, the kolibri_plugin.py file describes most of the configuration for the single-page app. In particular, this includes the base Django HTML template to return (with an empty <body>), the URL at which the app is exposed, and the javascript entry file which is run on load.

On the client-side, the app creates a single KolibriModule object in the entry file (conventionally app.js) and registers this with the core app, a global variable called kolibriGlobal. The Kolibri Module then mounts single root component to the HTML returned by the server, which recursively contains all additional components, html and logic.

Defining a New Kolibri Module

Note

This section is mostly relevant if you are creating a new app or plugin. If you are just creating new components, you don’t need to do this.

A Kolibri Module is initially defined in Python by sub-classing the WebpackBundleHook class (in kolibri.core.webpack.hooks). The hook defines the JS entry point (conventionally called app.js) where the KolibriModule subclass is instantiated, and where events and callbacks on the module are registered. These are defined in the events and once properties. Each defines key-value pairs of the name of an event, and the name of the method on the KolibriModule object. When these events are triggered on the Kolibri core JavaScript app, these callbacks will be called. (If the KolibriModule is registered for asynchronous loading, the Kolibri Module will first be loaded, and then the callbacks called when it is ready. See asset_loading for more information.)

All apps should extend the KolibriModule class found in kolibri/core/assets/src/kolibri_module.js.

The ready method will be automatically executed once the Module is loaded and registered with the Kolibri Core App. By convention, JavaScript is injected into the served HTML after the <rootvue> tag, meaning that this tag should be available when the ready method is called, and the root component (conventionally in vue/index.vue) can be mounted here.

Content Renderers

A special kind of Kolibri Module is dedicated to rendering particular content types. All content renderers should extend the ContentRendererModule class found in kolibri/core/assets/src/content_renderer_module.js. In addition, rather than subclassing the WebpackBundleHook class, content renderers should be defined in the Python code using the ContentRendererHook class defined in kolibri.content.hooks. In addition to the standard options for the WebpackBundleHook, the ContentRendererHook also accepts a json file defining the content types that it renders:

.. automodule:: kolibri.content.hooks
members:
noindex:

The ContentRendererModule class has one required property getRendererComponent which should return a Vue component that wraps the content rendering code. This component will be passed defaultFile, files, supplementaryFiles, and thumbnailFiles props, defining the files associated with the piece of content.

{
  props: [
    'defaultFile',
    'files',
  ]
};

In order to log data about users viewing content, the component should emit startTracking, updateProgress, and stopTracking events, using the Vue $emit method. startTracking and stopTracking are emitted without any arguments, whereas updateProgress should be emitted with a single value between 0 and 1 representing the current proportion of progress on the content.

this.$emit('startTracking');
this.$emit('stopTracking');
this.$emit('updateProgress', 0.25);

For content that has assessment functionality two additional props will be passed: itemId and answerState. itemId is a unique identifier for that content for a particular question in the assessment, answerState is passed to prefill an answer (one that has been previously given on an exam, or for a coach to preview a learner’s given answers). The answer renderer should also define a checkAnswer method in its component methods, this method should return an object with the following keys: correct, answerState, and simpleAnswer - describing the correctness, an object describing the answer that can be used to reconstruct it within the renderer, and a simple, human readable answer. If no valid answer is given, null should be returned. In addition to the base content renderer events, assessment items can also emit a hintTaken event to indicate that the user has taken a hint in the assessment, an itemError event to indicate that there has been an error in rendering the requested question corresponding to the itemId, and an interaction event that indicates a user has interacted with the assessment.

{
  props: [
    'defaultFile',
    'files',
    'itemId',
    'answerState',
  ],
  methods: {
    checkAnswer() {
      return {
        correct: true,
        answerState: {
          answer: 81,
          working: '3^2 = 3 * 3',
        },
        simpleAnswer: '81',
      };
    },
  },
};

Shared Core Functionality

Kolibri provides a set of shared “core” functionality – including components, styles, and helper logic, and libraries – which can be re-used across apps and plugins.

JS Libraries

The following libraries are available globally, in all module code:

  • vue - the Vue.js object
  • vuex - the Vuex object
  • logging - our wrapper around the loglevel logging module
  • core-base - a shared base Vue.js component (core-base.vue)

And many others. The complete specification for commonly shared modules can be found in kolibri/core/assets/src/core-app/apiSpec.js - this object defines which modules are imported into the core object. If the module in question has the ‘requireName’ attribute set on the core specification, then it can be used in code with a standard CommonJS-style require statement - e.g.:

const vue = require('kolibri.lib.vue');
const coreBase = require('kolibri.coreVue.components.coreBase');

Adding additional globally-available objects is relatively straightforward due to the plugin and webpack build system.

To expose something on the core app, add a key to the object in apiSpec.js which maps to an object with the following keys:

modulePath: {
    module: require('module-name'),
  }

This module would now be available for import anywhere with the following statement:

const MODULE = require('kolibri.modulePath');

For better organisation of the Core API specification, modules can also be attached at arbitrarily nested paths:

modulePath: {
    nestedPath: {
      module: require('module-name'),
    }
  }

This module would now be available for import anywhere with the following statement:

const MODULE = require('kolibri.modulePath.nestedPath');

For convenience (and to prevent accidental imports), 3rd party (NPM) modules installed in node_modules can be required by their usual name also:

const vue = require('vue');
Bootstrapped Data

The kolibriGlobal object is also used to bootstrap data into the JS app, rather than making unnecessary API requests.

For example, we currently embellish the kolibriGlobal object with a urls object. This is defined by Django JS Reverse and exposes Django URLs on the client side. This will primarily be used for accessing API Urls for synchronizing with the REST API. See the Django JS Reverse documentation for details on invoking the Url.

Styling

For shared styles, two mechanisms are provided:

  • The core-theme.styl file provides values for some globally-relevant Stylus variables. These variables can be used in any component’s <style> block by adding the line @require '~core-theme.styl'.
  • The core-global.styl file is always inserted into the <head> after normalize.css and provides some basic styling to global elements
Additional Functionality

These methods are also publicly exposed methods of the core app:

kolibriGlobal.register_kolibri_module_async   // Register a Kolibri module for asynchronous loading.
kolibriGlobal.register_kolibri_module_sync    // Register a Kolibri module once it has loaded.
kolibriGlobal.stopListening                   // Unbind an event/callback pair from triggering.
kolibriGlobal.emit                            // Emit an event, with optional args.

Unit Testing

Unit testing is carried out using Mocha. All JavaScript code should have unit tests for all object methods and functions.

Tests are written in JavaScript, and placed in the ‘assets/test’ folder. An example test is shown below:

var assert = require('assert');

var SearchModel = require('../src/search/search_model.js');

describe('SearchModel', function() {
  describe('default result', function() {
    it('should be empty an empty array', function () {
      var test_model = new SearchModel();
      assert.deepEqual(test_model.get("result"), []);
    });
  });
});

Vue.js components can also be tested. The management plugin contains an example (kolibri/plugins/management/assets/test/management.js) where the component is bound to a temporary DOM node, changes are made to the state, and assertions are made about the new component structure.

Adding Dependencies

Dependencies are tracked using yarn - see the docs here.

We distinguish development dependencies from runtime dependencies, and these should be installed as such using yarn add --dev [dep] or yarn add [dep], respectively. Your new dependency should now be recorded in package.json, and all of its dependencies should be recorded in yarn.lock.

Individual plugins can also have their own package.json and yarn.lock for their own dependencies. Running yarn install will also install all the dependencies for each activated plugin (inside a node_modules folder inside the plugin itself). These dependencies will only be available to that plugin at build time. Dependencies for individual plugins should be added from within the root directory of that particular plugin.

To assist in tracking the source of bloat in our codebase, the command yarn run bundle-stats is available to give a full readout of the size that uglified packages take up in the final Javascript code.

In addition, a plugin can have its own webpack.config.js for plugin specific webpack configuration (loaders, plugins, etc.). These options will be merged with the base options using webpack-merge.

Server-client communication

Server API

The Kolibri server represents data as Django Models. These models are defined in models.py files, which can be found in the folders of the different Django apps/plugins.

In Django, Model data are usually exposed to users through webpages that are generated by the Django server. To make the data available to the Kolibri client, which is a single-page app, the Models are exposed as JSON data through a REST API provided by the Django REST Framework (DRF).

In the api.py files, Django REST framework ViewSets are defined which describe how the data is made available through the REST API. Each ViewSet also requires a defined Serializer, which describes the way in which the data from the Django model is serialized into JSON and returned through the REST API. Additionally, optional filters can be applied to the ViewSet which will allow queries to filter by particular features of the data (for example by a field) or by more complex constraints, such as which group the user associated with the data belongs to. Permissions can be applied to a ViewSet, allowing the API to implicitly restrict the data that is returned, based on the currently logged in user.

Finally, in the api_urls.py file, the ViewSets are given a name (through the base_name keyword argument), which sets a particular URL namespace, which is then registered and exposed when the Django server runs. Sometimes, a more complex URL scheme is used, as in the content core app, where every query is required to be prefixed by a channel id (hence the <channel_id> placeholder in that route’s regex pattern)

api_urls.py
router = routers.SimpleRouter()
router.register('content', ChannelMetadataCacheViewSet, base_name="channel")

content_router = routers.SimpleRouter()
content_router.register(r'contentnode', ContentNodeViewset, base_name='contentnode')
content_router.register(r'file', FileViewset, base_name='file')

urlpatterns = [
    url(r'^', include(router.urls)),
    url(r'^content/(?P<channel_id>[^/.]+)/', include(content_router.urls)),
]

To explore the server REST APIs, visit /api_explorer/ on the Kolibri server while running with developer settings.

Client Resource Layer

To access this REST API in the frontend Javascript code, an abstraction layer has been written to reduce the complexity of inferring URLs, caching resources, and saving data back to the server.

Resources

In order to access a particular REST API endpoint, a Javascript Resource has to be defined, an example is shown here

channel.js
const Resource = require('kolibri.lib.apiResource').Resource;

class ChannelResource extends Resource {
  static resourceName() {
    return 'channel';
  }
}

module.exports = ChannelResource;

Here, the resourceName static method must return 'channel' in order to match the base_name assigned to the /content endpoint in api_urls.py.

However, in the case of a more complex endpoint, where arguments are required to form the URL itself (such as in the contentnode endpoints above) - we can add additional required arguments with the resourceIdentifiers static method return value

contentNode.js
const Resource = require('kolibri.lib.apiResource').Resource;

class ContentNodeResource extends Resource {
  static resourceName() {
    return 'contentnode';
  }
  static idKey() {
    return 'pk';
  }
  static resourceIdentifiers() {
    // because ContentNode resources are accessed via
    // /api/content/<channel_id>/contentnode/<pk>
    return [
      'channel_id',
    ];
  }
}

module.exports = ContentNodeResource;

If this resource is part of the core app, it can be added to a global registry of resources inside kolibri/core/assets/src/api-resources/index.js. Otherwise, it can be instantiated as needed, such as in the coach reports module

const ContentSummaryResourceConstructor = require('./apiResources/contentSummary');
const ContentSummaryResource = new ContentSummaryResourceConstructor(coreApp);

First the constructor is imported from the require file, and then an instance is created - with a reference to the Kolibri core app module passed as the only argument.

Models

The instantiated Resource can then be queried for client side representations of particular information. For a representation of a single server side Django model, we can request a Model from the Resource, using getModel

// corresponds to resource address /api/content/<channelId>/contentnode/<id>
const contentModel = ContentNodeResource.getModel(id, { channel_id: channelId });

The first argument is the database id (primary key) for the model, while the second argument defines any additional required resourceIdentifiers that we need to build up the URL.

We now have a reference for a representation of the data on the server. To ensure that it has data from the server, we can call .fetch on it which will resolve to an object representing the data

contentModel.fetch().then((data) => {
  logging.info('This is the model data: ', data);
});

The fetch method returns a Promise which resolves when the data has been successfully retrieved. This may have been due to a round trip call to the REST API, or, if the data has already been previously returned, then it will skip the call to the REST API and return a cached copy of the data.

If you want to pass additional GET parameters to the REST API (to only return a limited set of fields, for example), then you can pass GET parameters in the first argument

contentModel.fetch({ title: true }).then((data) => {
  logging.info('This is the model data: ', data);
});

If it is important to get data that has not been cached, you can call the fetch method with a force parameter

contentModel.fetch({}, true).then((data) => {
  logging.info('This is definitely the most up to date model data: ', data);
});
Collections

For particular views on a data table (which could range from ‘show me everything’ to ‘show me all content nodes with titles starting with “p”’) - Collections are used. Collections are a cached view onto the data table, which are populated by Models - so if a Model that has previously been fetched from the server by a Collection is requested from getModel, it is already cachced.

The first argument defines any additional required resourceIdentifiers that we need to build up the URL, while the second argument defines the GET parameters that are used to define the filters to be applied to the data and hence the subset of the data that the Collection represents.

We now have a reference for a representation of this data on the server. To ensure that it has data from the server, we can call fetch on it, this will resolve to an array of the returned data objects

contentCollection.fetch().then((dataArray) => {
  logging.info('This is the model data: ', dataArray);
});

The fetch method returns a Promise which resolves when the data has been successfully retrieved. This may have been due to a round trip call to the REST API, or, if the data has already been previously returned, then it will skip the call to the REST API and return a cached copy of the data.

If you want to pass additional GET parameters to the REST API (to only return a limited set of fields, for example), then you can pass GET parameters in the first argument

// GET /api/content/<channelId>/contentnode/?popular=1&title=true
contentCollection.fetch({ title: true }).then((dataArray) => {
  logging.info('This is the model data: ', dataArray);
});

If it is important to get data that has not been cached, you can call the fetch method with a force parameter

contentCollection.fetch({}, true).then((dataArray) => {
  logging.info('This is the model data: ', dataArray);
});

Data Flow Diagram

_images/full_stack_data_flow.svg

Content database module

This is a core module found in kolibri/Content.

Concepts and Definitions

ContentNode

High level abstraction for prepresenting different content kinds, such as Topic, Video, Audio, Exercise, Document, and can be easily extended to support new content kinds. With multiple ContentNode objects, it supports grouping, arranging them in tree structure, and symmetric and asymmetric relationship between two ContentNode objects.

File

A Django model that is used to store details about the source file, such as what language it supports, how big is the size, which format the file is and where to find the source file.

ContentDB Diagram
_images/content_distributed_db.png
  • PK = Primary Key
  • FK = Foreign Key
  • M2M = ManyToManyField
ContentTag

This model is used to establish a filtering system for all ContentNode objects.

ChannelMetadata

A Django model in each content database that stores the database readable names, description and author for each channel.

ChannelMetadataCache

This class stores the channel metadata cached/denormed into the default database.

Implementation Details and Workflows

To achieve using separate databases for each channel and being able to switch channels dynamically, the following data structure and utility functions have been implemented.

ContentDBRoutingMiddleware

This middleware will be applied to every request, and will dynamically select a database based on the channel_id. If a channel ID was included in the URL, it will ensure the appropriate content DB is used for the duration of the request. (Note: set_active_content_database is thread-local, so this shouldn’t interfere with other parallel requests.)

For example, this is how the client side dynamically requests data from a specific channel:

>>> localhost:8000/api/content/<channel_1_id>/contentnode

this will respond with all the contentnode data stored in database <channel_1_id>.sqlite3

>>> localhost:8000/api/content/<channel_2_id>/contentnode

this will respond with all the contentnode data stored in database <channel_2_id>.sqlite3

get_active_content_database

A utility function to retrieve the temporary thread-local variable that using_content_database sets

set_active_content_database

A utility function to set the temporary thread-local variable

using_content_database

A decorator and context manager to do queries on a specific content DB.

Usage as a context manager:

from models import ContentNode

with using_content_database("nalanda"):
    objects = ContentNode.objects.all()
    return objects.count()

Usage as a decorator:

from models import ContentNode

@using_content_database('nalanda')
def delete_all_the_nalanda_content():
    ContentNode.objects.all().delete()
ContentDBRouter

A router that decides what content database to read from based on a thread-local variable.

ContentNode

ContentNode is implemented as a Django model that inherits from two abstract classes, MPTTModel and ContentDatabaseModel. django-mptt’s MPTTModel, which allows for efficient traversal and querying of the ContentNode tree. ContentDatabaseModel is used as a marker so that the content_db_router knows to query against the content database only if the model inherits from ContentDatabaseModel.

The tree structure is established by the parent field that is a foreign key pointing to another ContentNode object. You can also create a symmetric relationship using the related field, or an asymmetric field using the is_prerequisite field.

File

The File model also inherits from ContentDatabaseModel.

To find where the source file is located, the class method get_url uses the checksum field and settings.CONTENT_STORAGE_DIR to calculate the file path. Every source file is named based on its MD5 hash value (this value is also stored in the checksum field) and stored in a namespaced folder under the directory specified in settings.CONTENT_STORAGE_DIR. Because it’s likely to have thousands of content files, and some filesystems cannot handle a flat folder with a large number of files very well, we create namespaced subfolders to improve the performance. So the eventual file path would look something like:

/home/user/.kolibri/content/storage/9/8/9808fa7c560b9801acccf0f6cf74c3ea.mp4

As you can see, it is fine to store your content files outside of the kolibri project folder as long as you set the settings.CONTENT_STORAGE_DIR accordingly.

The front-end will then use the extension field to decide which content player should be used. When the supplementary field’s value is True, that means this File object isn’t necessary and can display the content without it. For example, we will mark caption (subtitle) file as supplementary.

Content Constants

A Python module that stores constants for the kind field in ContentNode model and the preset field and extension field in File model.

Workflows

There are two workflows we currently designed to handle content UI rendering and content playback rendering

  • Content UI Rendering
  1. Start with a ContentNode object.
  2. Get the associated File object that has the thumbnail field being True.
  3. Get the thumbnail image by calling this File’s get_url method.
  4. Determine the template using the kind field of this ContentNode object.
  5. Renders the template with the thumbnail image.
  • Content Playback Rendering
  1. Start with a ContentNode object.
  2. Retrieve a queryset of associated File objects that are filtered by the preset.
  3. Use the thumbnail field as a filter on this queryset to get the File object and call this File object’s get_url method to get the source file (the thumbnail image)
  4. Use the supplementary field as a filter on this queryset to get the “supplementary” File objects, such as caption (subtitle), and call these File objects’ get_url method to get the source files.
  5. Use the supplementary field as a filter on this queryset to get the essential File object. Call its get_url method to get the source file and use its extension field to choose the content player.
  6. Play the content.

API Methods

class kolibri.content.api.ChannelMetadataViewSet(**kwargs)[source]
get_queryset()[source]

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ChannelMetadataSerializer

class kolibri.content.api.ContentNodeFileSizeViewSet(**kwargs)[source]
get_queryset()[source]

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ContentNodeGranularSerializer

class kolibri.content.api.ContentNodeGranularViewset(**kwargs)[source]
get_queryset()[source]

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ContentNodeGranularSerializer

class kolibri.content.api.ContentNodeProgressViewset(**kwargs)[source]
get_queryset()[source]

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ContentNodeProgressSerializer

class kolibri.content.api.ContentNodeViewset(**kwargs)[source]
get_object(prefetch=True)[source]

Returns the object the view is displaying. You may want to override this if you need to provide non-standard queryset lookups. Eg if objects are referenced using multiple keyword arguments in the url conf.

get_queryset(prefetch=True)[source]

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

pagination_class

alias of OptionalPageNumberPagination

serializer_class

alias of kolibri.content.serializers.ContentNodeSerializer

class kolibri.content.api.FileViewset(**kwargs)[source]
get_queryset()[source]

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

pagination_class

alias of OptionalPageNumberPagination

serializer_class

alias of kolibri.content.serializers.FileSerializer

class kolibri.content.api.OptionalPageNumberPagination[source]

Pagination class that allows for page number-style pagination, when requested. To activate, the page_size argument must be set. For example, to request the first 20 records: ?page_size=20&page=1

class kolibri.content.api.RemoteChannelViewSet(**kwargs)[source]
list(request, *args, **kwargs)[source]

Gets metadata about all public channels on kolibri studio.

retrieve(request, pk=None)[source]

Gets metadata about a channel through a token or channel id.

API endpoints

request specific content:

>>> localhost:8000/api/content/<channel_id>/contentnode/<content_id>

search content:

>>> localhost:8000/api/content/<channel_id>/contentnode/?search=<search words>

request specific content with specified fields:

>>> localhost:8000/api/content/<channel_id>/contentnode/<content_id>/?fields=pk,title,kind

request paginated contents

>>> localhost:8000/api/content/<channel_id>/contentnode/?page=6&page_size=10

request combines different usages

>>> localhost:8000/api/content/<channel_id>/contentnode/?fields=pk,title,kind,instance_id,description,files&page=6&page_size=10&search=wh

Models

This is one of the Kolibri core components, the abstract layer of all contents. To access it, please use the public APIs in api.py

The ONLY public object is ContentNode

class kolibri.content.models.AssessmentMetaData(*args, **kwargs)[source]

A model to describe additional metadata that characterizes assessment behaviour in Kolibri. This model contains additional fields that are only revelant to content nodes that probe a user’s state of knowledge and allow them to practice to Mastery. ContentNodes with this metadata may also be able to be used within quizzes and exams.

Parameters:
  • id (UUIDField) – Id
  • contentnode_id (ForeignKey to ~) – Contentnode
  • assessment_item_ids (JSONField) – Assessment item ids
  • number_of_assessments (IntegerField) – Number of assessments
  • mastery_model (JSONField) – Mastery model
  • randomize (BooleanField) – Randomize
  • is_manipulable (BooleanField) – Is manipulable
exception DoesNotExist
exception MultipleObjectsReturned
class kolibri.content.models.ChannelMetadata(*args, **kwargs)[source]

Holds metadata about all existing content databases that exist locally.

Parameters:
  • id (UUIDField) – Id
  • name (CharField) – Name
  • description (CharField) – Description
  • author (CharField) – Author
  • version (IntegerField) – Version
  • thumbnail (TextField) – Thumbnail
  • last_updated (DateTimeTzField) – Last updated
  • min_schema_version (CharField) – Min schema version
  • root_id (ForeignKey to ~) – Root
exception DoesNotExist
exception MultipleObjectsReturned
class kolibri.content.models.ContentNode(*args, **kwargs)[source]

The top layer of the contentDB schema, defines the most common properties that are shared across all different contents. Things it can represent are, for example, video, exercise, audio or document…

Parameters:
  • id (UUIDField) – Id
  • parent_id (TreeForeignKey to ~) – Parent
  • license_name (CharField) – License name
  • license_description (CharField) – License description
  • title (CharField) – Title
  • content_id (UUIDField) – Content id
  • channel_id (UUIDField) – Channel id
  • description (CharField) – Description
  • sort_order (FloatField) – Sort order
  • license_owner (CharField) – License owner
  • author (CharField) – Author
  • kind (CharField) – Kind
  • available (BooleanField) – Available
  • stemmed_metaphone (CharField) – Stemmed metaphone
  • lang_id (ForeignKey to ~) – Lang
  • lft (PositiveIntegerField) – Lft
  • rght (PositiveIntegerField) – Rght
  • tree_id (PositiveIntegerField) – Tree id
  • level (PositiveIntegerField) – Level
  • has_prerequisite (ManyToManyField) – Has prerequisite
  • related (ManyToManyField) – Related
  • tags (ManyToManyField) – Tags
exception DoesNotExist
exception MultipleObjectsReturned
get_descendant_content_ids()[source]

Retrieve a queryset of content_ids for non-topic content nodes that are descendants of this node.

class kolibri.content.models.ContentTag(id, tag_name)[source]
Parameters:
  • id (UUIDField) – Id
  • tag_name (CharField) – Tag name
exception DoesNotExist
exception MultipleObjectsReturned
class kolibri.content.models.File(*args, **kwargs)[source]

The second to bottom layer of the contentDB schema, defines the basic building brick for content. Things it can represent are, for example, mp4, avi, mov, html, css, jpeg, pdf, mp3…

Parameters:
  • id (UUIDField) – Id
  • local_file_id (ForeignKey to ~) – Local file
  • available (BooleanField) – Available
  • contentnode_id (ForeignKey to ~) – Contentnode
  • preset (CharField) – Preset
  • lang_id (ForeignKey to ~) – Lang
  • supplementary (BooleanField) – Supplementary
  • thumbnail (BooleanField) – Thumbnail
  • priority (IntegerField) – Priority
exception DoesNotExist
exception MultipleObjectsReturned
get_download_filename()[source]

Return a valid filename to be downloaded as.

get_download_url()[source]

Return the download url.

get_preset()[source]

Return the preset.

class kolibri.content.models.Language(id, lang_code, lang_subcode, lang_name, lang_direction)[source]
Parameters:
  • id (CharField) – Id
  • lang_code (CharField) – Lang code
  • lang_subcode (CharField) – Lang subcode
  • lang_name (CharField) – Lang name
  • lang_direction (CharField) – Lang direction
exception DoesNotExist
exception MultipleObjectsReturned
class kolibri.content.models.LocalFile(*args, **kwargs)[source]

The bottom layer of the contentDB schema, defines the local state of files on the device storage.

Parameters:
  • id (CharField) – Id
  • extension (CharField) – Extension
  • available (BooleanField) – Available
  • file_size (IntegerField) – File size
exception DoesNotExist
exception MultipleObjectsReturned
get_storage_url()[source]

Return a url for the client side to retrieve the content file. The same url will also be exposed by the file serializer.

class kolibri.content.models.UUIDField(*args, **kwargs)[source]

Adaptation of Django’s UUIDField, but with 32-char hex representation as Python representation rather than a UUID instance.

deconstruct()[source]

Returns enough information to recreate the field as a 4-tuple:

  • The name of the field on the model, if contribute_to_class has been run
  • The import path of the field, including the class: django.db.models.IntegerField This should be the most portable version, so less specific may be better.
  • A list of positional arguments
  • A dict of keyword arguments

Note that the positional or keyword arguments must contain values of the following types (including inner values of collection types):

  • None, bool, str, unicode, int, long, float, complex, set, frozenset, list, tuple, dict
  • UUID
  • datetime.datetime (naive), datetime.date
  • top-level classes, top-level functions - will be referenced by their full import path
  • Storage instances - these have their own deconstruct() method

This is because the values here must be serialized into a text format (possibly new Python code, possibly JSON) and these are the only types with encoding handlers defined.

There’s no need to return the exact way the field was instantiated this time, just ensure that the resulting field is the same - prefer keyword arguments over positional ones, and omit parameters with their default values.

get_db_prep_value(value, connection, prepared=False)[source]

Returns field’s value prepared for interacting with the database backend.

Used by the default implementations of get_db_prep_save``and `get_db_prep_lookup`

to_python(value)[source]

Converts the input value into the expected Python data type, raising django.core.exceptions.ValidationError if the data can’t be converted. Returns the converted value. Subclasses should override this.

Users, auth, and permissions module

This is a core module found in kolibri/auth.

Concepts and Definitions

Facility

All user data (accounts, logs, ratings, etc) in Kolibri are associated with a particular “Facility”. A Facility is a grouping of users who are physically co-located, and who generally access Kolibri from the same server on a local network, for example in a school, library, or community center. Collectively, all the data associated with a particular Facility are referred to as a “Facility Dataset”.

Users

There are two kinds of users: FacilityUser and DeviceOwner. A FacilityUser is associated with a particular Facility, and the user’s account and data may be synchronized across multiple devices. A DeviceOwner account is not associated with a particular Facility, but is specific to one device, and is never synchronized across multiple devices. A DeviceOwner is like a superuser, and has permissions to modify any data on her own device, whereas a FacilityUser only has permissions for some subset of data from their own Facility Dataset (as determined in part by the roles they possess; see below).

Collections

Collections are hierarchical groups of users, used for grouping users and making decisions about permissions. Users can have roles for one or more Collections, by way of obtaining Roles associated with those Collections. Collections can belong to other Collections, and user membership in a collection is conferred through Membership. Collections are subdivided into several pre-defined levels: Facility, Classroom, and LearnerGroup, as illustrated here:

_images/uap_collection_hierarchy.svg

In this illustration, Facility X contains two Classrooms, Class A and Class B. Class A contains two LearnerGroups, Group Q and Group R.

Membership

A FacilityUser (but not a DeviceOwner) can be marked as a member of a Collection through a Membership object. Being a member of a Collection also means being a member of all the Collections above that Collection in the hierarchy. Thus, in the illustration below, Alice is directly associated with Group Q through a Membership object, which makes her a member of Group Q. As Group Q is contained within Class A, which is contained within Facility X, she is also implicitly a member of both those collections.

_images/uap_membership_diagram.svg

Note also that a FacilityUser is always implicitly a member of the Facility with which it is associated, even if it does not have any Membership objects.

Roles

Another way in which a FacilityUser can be associated with a particular Collection is through a Role object, which grants the user a role with respect to the Collection and all the collections below it. A Role object also stores the “kind” of the role (currently, one of “admin” or “coach”), which affects what permissions the user gains through the Role.

To illustrate, consider the example in the following figure:

_images/uap_role_membership_diagram.svg

The figure shows a Role object linking Bob with Class A, and the Role is marked with kind “coach”, which we can informally read as “Bob is a coach for Class A”. We consider user roles to be “downward-transitive” (meaning if you have a role for a collection, you also have that role for descendents of that collection). Thus, in our example, we can say that “Bob is also a coach for Group Q”. Furthermore, as Alice is a member of Group Q, we can say that “Bob is a coach for Alice”.

Role-Based Permissions

As a lot of Facility Data in Kolibri is associated with a particular FacilityUser, for many objects we can concisely define a requesting user’s permissions in terms of his or her roles for the object’s associated User. For example, if a ContentLog represents a particular FacilityUser’s interaction with a piece of content, we might decide that another FacilityUser can view the ContentLog if she is a coach (has the coach role) for the user. In our scenario above, this would mean that Bob would have read permissions for a ContentLog for which “user=Alice”, by virtue of having the coach role for Alice.

Some data may not be related to a particular user, but rather with a Collection (e.g. the Collection object itself, settings for a Collection, or content assignments for a Collection). Permissions for these objects can be defined in terms of the role the requesting User has with respect to the object’s associated Collection. So, for example, we might allow Bob to assign content to Class A on the basis of him having the “coach” role for Class A.

Permission Levels

As we are constructing a RESTful API for accessing data within Kolibri, the core actions for which we need to define permissions are the CRUD operations (Create, Read, Update, Delete). As Create, Update, and Delete permissions often go hand in hand, we can collectively refer to them as “Write Permissions”.

_images/uap_crud_permissions.svg

Implementation Details

Collections

A Collection is implemented as a Django model that inherits from django-mptt’s MPTTModel, which allows for efficient traversal and querying of the collection hierarchy. For convenience, the specific types of collections – Facility, Classroom, and LearnerGroup – are implemented as _proxy models of the main Collection model. There is a kind field on Collection that allows us to distinguish between these types, and the ModelManager for the proxy models returns only instances of the matching kind.

From a Collection instance, you can traverse upwards in the tree with the parent field, and downwards via the children field (which is a reverse RelatedManager for the parent field):

>>> my_classroom.parent
<Collection: "Facility X" (facility)>

>>> my_facility.children.all()
[<Collection: "Class A" (classroom)>, <Collection: "Class B" (classroom)>]

Note that the above methods (which are provided by MPTTModel) return generic Collection instances, rather than specific proxy model instances. To retrieve parents and children as appropriate proxy models, use the helper methods provided on the proxy models, e.g.:

>>> my_classroom.get_facility()
<Facility: Facility X>

>>> my_facility.get_classrooms()
[<Classroom: Class A>, <Classroom: Class B>]
Facility and FacilityDataset

The Facility model (a proxy model for Collection, as described above) is special in that it has no parent; it is the root of a tree. A Facility model instance, and all other Facility Data associated with the Facility and its FacilityUsers, inherits from AbstractFacilityDataModel, which has a dataset field that foreign keys onto a common FacilityDataset instance. This makes it easy to check, for purposes of permissions or filtering data for synchronization, which instances are part of a particular Facility Dataset. The dataset field is automatically set during the save method, by calling the infer_dataset method, which must be overridden in every subclass of AbstractFacilityDataModel to return the dataset to associate with that instance.

Efficient Hierarchy Calculations

In order to make decisions about whether a user has a certain permission for an object, we need an efficient way to retrieve the set of roles the user has in relation to that object. This involves traversing the Role table, Collection hierarchy, and possibly the Membership table, but we can delegate most of the work to the database engine (and leverage efficient hierarchy lookups afforded by MPTT). The following algorithms and explanations will refer to the naming in the following diagram:

_images/uap_role_membership_queries.svg

In pseudocode, the query for “What Roles does Source User have in relation to Target User?” would be implemented in the following way:

Fetch all Roles with:
    User: Source User
    Collection: Ancestor Collection
For which there is a Membership with:
    User: Target User
    Collection: Descendant Collection
And where:
    Ancestor Collection is an ancestor of (or equal to) Descendant Collection

At the database level, this can be written in the following way, as a single multi-table SQL query:

SELECT DISTINCT
    source_role.kind
FROM
    collection_table AS ancestor_coll,
    collection_table AS descendant_coll,
    role_table,
    membership_table
WHERE
    role_table.user_id = {source_user_id} AND
    role_table.collection_id = ancestor_coll.id AND
    membership_table.user_id = {target_user_id}
    membership_table.collection_id = descendant_coll.id AND
    descendant_coll.lft BETWEEN ancestor_coll.lft AND ancestor_coll.rght AND
    descendant_coll.tree_id = ancestor_coll.tree_id;

Similarly, performing a queryset filter like “give me all ContentLogs associated with FacilityUsers for which Source User has an admin role” can be written as:

SELECT
    contentlog_table.*
FROM
    contentlog_table
WHERE EXISTS
    (SELECT
         *
     FROM
         collection_table AS ancestor_coll,
         collection_table AS descendant_coll,
         role_table,
         membership_table
     WHERE
         role_table.user_id = {source_user_id} AND
         role_table.collection_id = ancestor_coll.id AND
         membership_table.user_id = contentlog_table.user_id
         membership_table.collection_id = descendant_coll.id AND
         descendant_coll.lft BETWEEN ancestor_coll.lft AND ancestor_coll.rght AND
         descendant_coll.tree_id = ancestor_coll.tree_id
    )

Note the membership_table.user_id = contentlog_table.user_id condition, which links the role-membership-collection hierarchy subquery into the main query. We refer to this condition as the “anchor”.

To facilitate making queries that leverage the role-membership-collection hierarchy, without needing to write custom SQL each time, we have implemented a HierarchyRelationsFilter helper class. The class is instantiated by passing in a queryset, and then exposes a filter_by_hierarchy method that allows various parts of the role-membership-collection hierarchy to be constrained, and anchored back into the queryset’s main table. It then returns a filtered queryset (with appropriate conditions applied) upon which further filters or other queryset operations can be applied.

The signature for filter_by_hierarchy is:

def filter_by_hierarchy(self,
                        source_user=None,
                        role_kind=None,
                        ancestor_collection=None,
                        descendant_collection=None,
                        target_user=None):

With the exception of role_kind (which is either a string or list of strings, of role kinds), these parameters accept either:

  • A model instance (either a FacilityUser or a Collection subclass, as appropriate) or its ID
  • An F expression that anchors some part of the hierarchy back into the base queryset model (the simplest usage is just to put the name of a field from the base model in the F function, but you can also indirectly reference fields of related models, e.g. F("collection__parent"))

For example, the ContentLog query described above (“give me all ContentLogs associated with FacilityUsers for which Source User has an admin role”) can be implemented as:

contentlogs = HierarchyRelationsFilter(ContentLog.objects.all()).filter_by_hierarchy(
    source_user=my_source_user,  # specify the specific user to be the source user
    role_kind=role_kinds.ADMIN,  # make sure the Role is an admin role
    target_user=F("user"),  # anchor the target user to the "user" field of the ContentLog model
)
Managing Roles and Memberships

User and Collection models have various helper methods for retrieving and modifying roles and memberships:

  • To get all the members of a collection (including those of its descendant collections), use Collection.get_members().
  • To add or remove roles/memberships, use the add_role, remove_role, add_member, and remove_member methods of Collection (or the additional convenience methods, such as add_admin, that exist on the proxy models).
  • To check whether a user is a member of a Collection, use KolibriAbstractBaseUser.is_member_of (for DeviceOwner, this always returns False)
  • To check whether a user has a particular kind of role for a collection or another user, use the has_role_for_collection and has_role_for_user methods of KolibriAbstractBaseUser.
  • To list all role kinds a user has for a collection or another user, use the get_roles_for_collection and get_roles_for_user methods of KolibriAbstractBaseUser.
Encoding Permission Rules

We need to associate a particular set of rules with each model, to specify the permissions that users should have in relation to instances of that model. While not all models have the same rules, there are some broad categories of models that do share the same rules (e.g. ContentInteractionLog, ContentSummaryLog, and UserSessionLog – collectively, “User Log Data”). Hence, it is useful to encapsulate a permissions “class” that can be reused across models, and extended (through inheritance) if slightly different behavior is needed. These classes of permissions are defined as Python classes that inherit from kolibri.auth.permissions.base.BasePermissions, which defines the following overridable methods:

  • The following four Boolean (True/False) permission checks, corresponding to the “CRUD” operations: - user_can_create_object - user_can_read_object - user_can_update_object - user_can_delete_object
  • The queryset-filtering readable_by_user_filter method, which takes in a queryset and returns a queryset filtered down to just objects that should be readable by the user.
Associating Permissions with Models

A model is associated with a particular permissions class through a “permissions” attribute defined on the top level of the model class, referencing an instance of a Permissions class (a class that subclasses BasePermissions). For example, to specify that a model ContentSummaryLog should draw its permissions rules from the UserLogPermissions class, modify the model definition as follows:

class ContentSummaryLog(models.Model):

    permissions = UserLogPermissions()

    <remainder of model definition>
Specifying Role-Based Permissions

Defining a custom Permissions class and overriding its methods allows for arbitrary logic to be used in defining the rules governing the permissions, but many cases can be covered by more constrained rule specifications. In particular, the rules for many models can be specified in terms of the role- based permissions system described above. A built-in subclass of BasePermissions, called RoleBasedPermissions, makes this easy. Creating an instance of RoleBasedPermissions involves passing in the following parameters:

  • Tuples of role kinds that should be granted each of the CRUD permissions, encoded in the following parameters: can_be_created_by, can_be_read_by, can_be_updated_by, can_be_deleted_by.
  • The target_field parameter that determines the “target” object for the role-checking; this should be the name of a field on the model that foreign keys either onto a FacilityUser or a Collection. If the model we’re checking permissions for is itself the target, then target_field may be ".".

An example, showing that read permissions should be granted to a coach or admin for the user referred to by the model’s “user” field. Similarly, write permissions should only be available to an admin for the user:

class UserLog(models.Model):

    permissions = RoleBasedPermissions(
        target_field="user",
        can_be_created_by=(role_kinds.ADMIN,),
        can_be_read_by=(role_kinds.COACH, role_kinds.ADMIN),
        can_be_updated_by=(role_kinds.ADMIN,),
        can_be_deleted_by=(role_kinds.ADMIN,),
    )

    <remainder of model definition>
Built-in Permissions Classes

Some common rules are encapsulated by the permissions classes in kolibri.auth.permissions.general. These include:

  • IsOwn: only allows access to the object if the object belongs to the requesting user (in other words, if the object has a specific field, field_name, that foreign keys onto the user)
  • IsFromSameFacility: only allows access to object if user is associated with the same facility as the object
  • IsSelf: only allows access to the object if the object is the user

A general pattern with these provided classes is to allow an argument called read_only, which means that rather than allowing both write (create, update, delete) and read permissions, they will only grant read permission. So, for example, IsFromSameFacility(read_only=True) will allow any user from the same facility to read the model, but not to write to it, whereas IsFromSameFacility(read_only=False) or IsFromSameFacility() would allow both.

Combining Permissions Classes

In many cases, it may be necessary to combine multiple permission classes together to define the ruleset that you want. This can be done using the Boolean operators | (OR) and & (AND). So, for example, IsOwn(field_name="user") | IsSelf() would allow access to the model if either the model has a foreign key named “user” that points to the user, or the model is itself the user model. Combining two permission classes with &, on the other hand, means both classes must return True for a permission to be granted. Note that permissions classes combined in this way still support the readable_by_user_filter method, returning a queryset that is either the union (for |) or intersection (&) of the querysets that were returned by each of the permissions classes.

Checking Permissions

Checking whether a user has permission to perform a CRUD operation on an object involves calling the appropriate methods on the KolibriAbstractBaseUser (FacilityUser or DeviceOwner) instance. For instance, to check whether request.user has delete permission for ContentSummaryLog instance log_obj, you could do:

if request.user.can_delete(log_obj):
    log_obj.delete()

Checking whether a user can create an object is slightly different, as you may not yet have an instance of the model. Instead, pass in the model class and a dict of the data that you want to create it with:

data = {"user": request.user, "content_id": "qq123"}
if request.user.can_create(ContentSummaryLog, data):
    ContentSummaryLog.objects.create(**data)

To efficiently filter a queryset so that it only includes records that the user should have permission to read (to make sure you’re not sending them data they shouldn’t be able to access), use the filter_readable method:

all_results = ContentSummaryLog.objects.filter(content_id="qq123")
permitted_results = request.user.filter_readable(all_results)

Note that for the DeviceOwner model, these methods will simply return True (or unfiltered querysets), as device owners are considered superusers. For the FacilityUser model, they defer to the permissions encoded in the permission object on the model class.

Using Kolibri Permissions with Django REST Framework

There are two classes that make it simple to leverage the permissions system described above within a Django REST Framework ViewSet, to restrict permissions appropriately on API endpoints, based on the currently logged-in user.

KolibriAuthPermissions is a subclass of rest_framework.permissions.BasePermission that defers to our KolibriAbstractBaseUser permissions interface methods for determining which object-level permissions to grant to the current user:

  • Permissions for ‘POST’ are based on request.user.can_create
  • Permissions for ‘GET’, ‘OPTIONS’ and ‘HEAD’ are based on request.user.can_read (Note that adding KolibriAuthPermissions only checks object-level permissions, and does not filter queries made against a list view; see KolibriAuthPermissionsFilter below)
  • Permissions for ‘PUT’ and ‘PATCH’ are based on request.user.can_update
  • Permissions for ‘DELETE’ are based on request.user.can_delete

KolibriAuthPermissions is a subclass of rest_framework.filters.BaseFilterBackend that filters list views to include only records for which the current user has read permissions. This only applies to ‘GET’ requests.

For example, to use the Kolibri permissions system to restrict permissions for an API endpoint providing access to a ContentLog model, you would do the following:

from kolibri.auth.api import KolibriAuthPermissions, KolibriAuthPermissionsFilter

class FacilityViewSet(viewsets.ModelViewSet):
    permission_classes = (KolibriAuthPermissions,)
    filter_backends = (KolibriAuthPermissionsFilter,)
    queryset = ContentLog.objects.all()
    serializer_class = ContentLogSerializer

Models

We have four main abstractions: Users, Collections, Memberships, and Roles.

Users represent people, like students in a school, teachers for a classroom, or volunteers setting up informal installations. A FacilityUser belongs to a particular facility, and has permissions only with respect to other data that is associated with that facility. FacilityUser accounts (like other facility data) may be synced across multiple devices.

Collections form a hierarchy, with Collections able to belong to other Collections. Collections are subdivided into several pre-defined levels (Facility > Classroom > LearnerGroup).

A FacilityUser (but not a DeviceOwner) can be marked as a member of a Collection through a Membership object. Being a member of a Collection also means being a member of all the Collections above that Collection in the hierarchy.

Another way in which a FacilityUser can be associated with a particular Collection is through a Role object, which grants the user a role with respect to the Collection and all the collections below it. A Role object also stores the “kind” of the role (currently, one of “admin” or “coach”), which affects what permissions the user gains through the Role.

class kolibri.auth.models.AbstractFacilityDataModel(*args, **kwargs)[source]

Base model for Kolibri “Facility Data”, which is data that is specific to a particular Facility, such as FacilityUsers, Collections, and other data associated with those users and collections.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

clean_fields(*args, **kwargs)[source]

Cleans all fields and raises a ValidationError containing a dict of all validation errors if any occur.

ensure_dataset(*args, **kwargs)[source]

If no dataset has yet been specified, try to infer it. If a dataset has already been specified, to prevent inconsistencies, make sure it matches the inferred dataset, otherwise raise a KolibriValidationError. If we have no dataset and it can’t be inferred, we raise a KolibriValidationError exception as well.

full_clean(*args, **kwargs)[source]

Calls clean_fields, clean, and validate_unique, on the model, and raises a ValidationError for any errors that occurred.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

save(*args, **kwargs)[source]

Saves the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

class kolibri.auth.models.Classroom(id, _morango_dirty_bit, _morango_source_id, _morango_partition, dataset, name, parent, kind, lft, rght, tree_id, level)[source]
Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • name (CharField) – Name
  • parent_id (TreeForeignKey to ~) – Parent
  • kind (CharField) – Kind
  • lft (PositiveIntegerField) – Lft
  • rght (PositiveIntegerField) – Rght
  • tree_id (PositiveIntegerField) – Tree id
  • level (PositiveIntegerField) – Level
exception DoesNotExist
exception MultipleObjectsReturned
get_facility()[source]

Gets the Classroom’s parent Facility.

Returns:A Facility instance.
get_learner_groups()[source]

Returns a QuerySet of LearnerGroups associated with this Classroom.

Returns:A LearnerGroup QuerySet.
save(*args, **kwargs)[source]

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.Collection(*args, **kwargs)[source]

Collections are hierarchical groups of FacilityUsers, used for grouping users and making decisions about permissions. FacilityUsers can have roles for one or more Collections, by way of obtaining Roles associated with those Collections. Collections can belong to other Collections, and user membership in a Collection is conferred through Memberships. Collections are subdivided into several pre-defined levels.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • name (CharField) – Name
  • parent_id (TreeForeignKey to ~) – Parent
  • kind (CharField) – Kind
  • lft (PositiveIntegerField) – Lft
  • rght (PositiveIntegerField) – Rght
  • tree_id (PositiveIntegerField) – Tree id
  • level (PositiveIntegerField) – Level
exception DoesNotExist
exception MultipleObjectsReturned
add_member(user)[source]

Create a Membership associating the provided user with this Collection. If the Membership object already exists, just return that, without changing anything.

Parameters:user – The FacilityUser to add to this Collection.
Returns:The Membership object (possibly new) that associates the user with the Collection.
add_role(user, role_kind)[source]

Create a Role associating the provided user with this collection, with the specified kind of role. If the Role object already exists, just return that, without changing anything.

Parameters:
  • user – The FacilityUser to associate with this Collection.
  • role_kind – The kind of role to give the user with respect to this Collection.
Returns:

The Role object (possibly new) that associates the user with the Collection.
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

clean_fields(*args, **kwargs)[source]

Cleans all fields and raises a ValidationError containing a dict of all validation errors if any occur.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

remove_member(user)[source]

Remove any Membership objects associating the provided user with this Collection.

Parameters:user – The FacilityUser to remove from this Collection.
Returns:True if a Membership was removed, False if there was no matching Membership to remove.
remove_role(user, role_kind)[source]

Remove any Role objects associating the provided user with this Collection, with the specified kind of role.

Parameters:
  • user – The FacilityUser to dissociate from this Collection (for the specific role kind).
  • role_kind – The kind of role to remove from the user with respect to this Collection.
save(*args, **kwargs)[source]

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.CollectionProxyManager[source]
get_queryset()[source]

Ensures that this manager always returns nodes in tree order.

class kolibri.auth.models.Facility(id, _morango_dirty_bit, _morango_source_id, _morango_partition, dataset, name, parent, kind, lft, rght, tree_id, level)[source]
Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • name (CharField) – Name
  • parent_id (TreeForeignKey to ~) – Parent
  • kind (CharField) – Kind
  • lft (PositiveIntegerField) – Lft
  • rght (PositiveIntegerField) – Rght
  • tree_id (PositiveIntegerField) – Tree id
  • level (PositiveIntegerField) – Level
exception DoesNotExist
exception MultipleObjectsReturned
ensure_dataset(*args, **kwargs)[source]

If no dataset has yet been specified, try to infer it. If a dataset has already been specified, to prevent inconsistencies, make sure it matches the inferred dataset, otherwise raise a KolibriValidationError. If we have no dataset and it can’t be inferred, we raise a KolibriValidationError exception as well.

get_classrooms()[source]

Returns a QuerySet of Classrooms under this Facility.

Returns:A Classroom QuerySet.
infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

save(*args, **kwargs)[source]

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.FacilityDataSyncableModel(*args, **kwargs)[source]
Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
class kolibri.auth.models.FacilityDataset(*args, **kwargs)[source]

FacilityDataset stores high-level metadata and settings for a particular Facility. It is also the model that all models storing facility data (data that is associated with a particular facility, and that inherits from AbstractFacilityDataModel) foreign key onto, to indicate that they belong to this particular Facility.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • description (TextField) – Description
  • location (CharField) – Location
  • preset (CharField) – Preset
  • learner_can_edit_username (BooleanField) – Learner can edit username
  • learner_can_edit_name (BooleanField) – Learner can edit name
  • learner_can_edit_password (BooleanField) – Learner can edit password
  • learner_can_sign_up (BooleanField) – Learner can sign up
  • learner_can_delete_account (BooleanField) – Learner can delete account
  • learner_can_login_with_no_password (BooleanField) – Learner can login with no password
exception DoesNotExist
exception MultipleObjectsReturned
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

class kolibri.auth.models.FacilityUser(*args, **kwargs)[source]

FacilityUser is the fundamental object of the auth app. These users represent the main users, and can be associated with a hierarchy of Collections through Memberships and Roles, which then serve to help determine permissions.

Parameters:
  • password (CharField) – Password
  • last_login (DateTimeField) – Last login
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • username (CharField) – Required. 30 characters or fewer. Letters and digits only
  • full_name (CharField) – Full name
  • date_joined (DateTimeTzField) – Date joined
  • facility_id (ForeignKey to ~) – Facility
exception DoesNotExist
exception MultipleObjectsReturned
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

can_create_instance(obj)[source]

Checks whether this user (self) has permission to create a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

In general, unless an instance has already been initialized, this method should not be called directly; instead, it should be preferred to call can_create.

Parameters:obj – An (unsaved) instance of a Django model, to check permissions for.
Returns:True if this user should have permission to create the object, otherwise False.
Return type:bool
can_delete(obj)[source]

Checks whether this user (self) has permission to delete a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to delete the object, otherwise False.
Return type:bool
can_manage_content

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

can_read(obj)[source]

Checks whether this user (self) has permission to read a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to read the object, otherwise False.
Return type:bool
can_update(obj)[source]

Checks whether this user (self) has permission to update a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to update the object, otherwise False.
Return type:bool
filter_readable(queryset)[source]

Filters a queryset down to only the elements that this user should have permission to read.

Parameters:queryset – A QuerySet instance that the filtering should be applied to.
Returns:Filtered QuerySet including only elements that are readable by this user.
get_roles_for_collection(coll)[source]

Determine all the roles this user has in relation to the specified Collection, and return a set containing the kinds of roles.

Parameters:coll – The target Collection for which this user has the roles.
Returns:The kinds of roles this user has with respect to the specified Collection.
Return type:set of kolibri.auth.constants.role_kinds.* strings
get_roles_for_user(user)[source]

Determine all the roles this user has in relation to the target user, and return a set containing the kinds of roles.

Parameters:user – The target user for which this user has the roles.
Returns:The kinds of roles this user has with respect to the target user.
Return type:set of kolibri.auth.constants.role_kinds.* strings
has_role_for_collection(kinds, coll)[source]

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified Collection.

Parameters:
  • kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
  • coll – The target Collection for which this user has the roles.
Returns:

True if this user has the specified role kind with respect to the target Collection, otherwise False.
Return type:

bool
has_role_for_user(kinds, user)[source]

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified user.

Parameters:
  • user – The user that is the target of the role (for which this user has the roles).
  • kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
Returns:

True if this user has the specified role kind with respect to the target user, otherwise False.
Return type:

bool
infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

is_member_of(coll)[source]

Determine whether this user is a member of the specified Collection.

Parameters:coll – The Collection for which we are checking this user’s membership.
Returns:True if this user is a member of the specified Collection, otherwise False.
Return type:bool
is_staff

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

is_superuser

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

class kolibri.auth.models.KolibriAbstractBaseUser(*args, **kwargs)[source]

Our custom user type, derived from AbstractBaseUser as described in the Django docs. Draws liberally from django.contrib.auth.AbstractUser, except we exclude some fields we don’t care about, like email.

This model is an abstract model, and is inherited by FacilityUser.

Parameters:
  • password (CharField) – Password
  • last_login (DateTimeField) – Last login
  • username (CharField) – Required. 30 characters or fewer. Letters and digits only
  • full_name (CharField) – Full name
  • date_joined (DateTimeTzField) – Date joined
can_create(Model, data)[source]

Checks whether this user (self) has permission to create an instance of Model with the specified attributes (data).

This method defers to the can_create_instance method, and in most cases should not itself be overridden.

Parameters:
  • Model – A subclass of django.db.models.Model
  • data – A dict of data to be used in creating an instance of the Model
Returns:

True if this user should have permission to create an instance of Model with the specified data, else False.
Return type:

bool
can_create_instance(obj)[source]

Checks whether this user (self) has permission to create a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

In general, unless an instance has already been initialized, this method should not be called directly; instead, it should be preferred to call can_create.

Parameters:obj – An (unsaved) instance of a Django model, to check permissions for.
Returns:True if this user should have permission to create the object, otherwise False.
Return type:bool
can_delete(obj)[source]

Checks whether this user (self) has permission to delete a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to delete the object, otherwise False.
Return type:bool
can_read(obj)[source]

Checks whether this user (self) has permission to read a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to read the object, otherwise False.
Return type:bool
can_update(obj)[source]

Checks whether this user (self) has permission to update a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to update the object, otherwise False.
Return type:bool
filter_readable(queryset)[source]

Filters a queryset down to only the elements that this user should have permission to read.

Parameters:queryset – A QuerySet instance that the filtering should be applied to.
Returns:Filtered QuerySet including only elements that are readable by this user.
get_roles_for(obj)[source]

Helper function that defers to get_roles_for_user or get_roles_for_collection based on the type of object passed in.

get_roles_for_collection(coll)[source]

Determine all the roles this user has in relation to the specified Collection, and return a set containing the kinds of roles.

Parameters:coll – The target Collection for which this user has the roles.
Returns:The kinds of roles this user has with respect to the specified Collection.
Return type:set of kolibri.auth.constants.role_kinds.* strings
get_roles_for_user(user)[source]

Determine all the roles this user has in relation to the target user, and return a set containing the kinds of roles.

Parameters:user – The target user for which this user has the roles.
Returns:The kinds of roles this user has with respect to the target user.
Return type:set of kolibri.auth.constants.role_kinds.* strings
has_role_for(kinds, obj)[source]

Helper function that defers to has_role_for_user or has_role_for_collection based on the type of object passed in.

has_role_for_collection(kinds, coll)[source]

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified Collection.

Parameters:
  • kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
  • coll – The target Collection for which this user has the roles.
Returns:

True if this user has the specified role kind with respect to the target Collection, otherwise False.
Return type:

bool
has_role_for_user(kinds, user)[source]

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified user.

Parameters:
  • user – The user that is the target of the role (for which this user has the roles).
  • kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
Returns:

True if this user has the specified role kind with respect to the target user, otherwise False.
Return type:

bool
is_member_of(coll)[source]

Determine whether this user is a member of the specified Collection.

Parameters:coll – The Collection for which we are checking this user’s membership.
Returns:True if this user is a member of the specified Collection, otherwise False.
Return type:bool
class kolibri.auth.models.KolibriAnonymousUser[source]

Custom anonymous user that also exposes the same interface as KolibriAbstractBaseUser, for consistency.

Parameters:
  • password (CharField) – Password
  • last_login (DateTimeField) – Last login
  • username (CharField) – Required. 30 characters or fewer. Letters and digits only
  • full_name (CharField) – Full name
  • date_joined (DateTimeTzField) – Date joined
can_create_instance(obj)[source]

Checks whether this user (self) has permission to create a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

In general, unless an instance has already been initialized, this method should not be called directly; instead, it should be preferred to call can_create.

Parameters:obj – An (unsaved) instance of a Django model, to check permissions for.
Returns:True if this user should have permission to create the object, otherwise False.
Return type:bool
can_delete(obj)[source]

Checks whether this user (self) has permission to delete a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to delete the object, otherwise False.
Return type:bool
can_read(obj)[source]

Checks whether this user (self) has permission to read a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to read the object, otherwise False.
Return type:bool
can_update(obj)[source]

Checks whether this user (self) has permission to update a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for.
Returns:True if this user should have permission to update the object, otherwise False.
Return type:bool
filter_readable(queryset)[source]

Filters a queryset down to only the elements that this user should have permission to read.

Parameters:queryset – A QuerySet instance that the filtering should be applied to.
Returns:Filtered QuerySet including only elements that are readable by this user.
get_roles_for_collection(coll)[source]

Determine all the roles this user has in relation to the specified Collection, and return a set containing the kinds of roles.

Parameters:coll – The target Collection for which this user has the roles.
Returns:The kinds of roles this user has with respect to the specified Collection.
Return type:set of kolibri.auth.constants.role_kinds.* strings
get_roles_for_user(user)[source]

Determine all the roles this user has in relation to the target user, and return a set containing the kinds of roles.

Parameters:user – The target user for which this user has the roles.
Returns:The kinds of roles this user has with respect to the target user.
Return type:set of kolibri.auth.constants.role_kinds.* strings
has_role_for_collection(kinds, coll)[source]

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified Collection.

Parameters:
  • kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
  • coll – The target Collection for which this user has the roles.
Returns:

True if this user has the specified role kind with respect to the target Collection, otherwise False.
Return type:

bool
has_role_for_user(kinds, user)[source]

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified user.

Parameters:
  • user – The user that is the target of the role (for which this user has the roles).
  • kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
Returns:

True if this user has the specified role kind with respect to the target user, otherwise False.
Return type:

bool
is_member_of(coll)[source]

Determine whether this user is a member of the specified Collection.

Parameters:coll – The Collection for which we are checking this user’s membership.
Returns:True if this user is a member of the specified Collection, otherwise False.
Return type:bool
class kolibri.auth.models.LearnerGroup(id, _morango_dirty_bit, _morango_source_id, _morango_partition, dataset, name, parent, kind, lft, rght, tree_id, level)[source]
Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • name (CharField) – Name
  • parent_id (TreeForeignKey to ~) – Parent
  • kind (CharField) – Kind
  • lft (PositiveIntegerField) – Lft
  • rght (PositiveIntegerField) – Rght
  • tree_id (PositiveIntegerField) – Tree id
  • level (PositiveIntegerField) – Level
exception DoesNotExist
exception MultipleObjectsReturned
get_classroom()[source]

Gets the LearnerGroup’s parent Classroom.

Returns:A Classroom instance.
save(*args, **kwargs)[source]

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.Membership(*args, **kwargs)[source]

A FacilityUser can be marked as a member of a Collection through a Membership object. Being a member of a Collection also means being a member of all the Collections above that Collection in the tree (i.e. if you are a member of a LearnerGroup, you are also a member of the Classroom that contains that LearnerGroup, and of the Facility that contains that Classroom).

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • user_id (ForeignKey to ~) – User
  • collection_id (TreeForeignKey to ~) – Collection
exception DoesNotExist
exception MultipleObjectsReturned
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.auth.models.Role(*args, **kwargs)[source]

A FacilityUser can have a role for a particular Collection through a Role object, which also stores the “kind” of the Role (currently, one of “admin” or “coach”). Having a role for a Collection also implies having that role for all sub-collections of that Collection (i.e. all the Collections below it in the tree).

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • user_id (ForeignKey to ~) – User
  • collection_id (TreeForeignKey to ~) – Collection
  • kind (CharField) – Kind
exception DoesNotExist
exception MultipleObjectsReturned
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

User log module

This is a core module found in kolibri/logger.

Concepts and Definitions

Content Interaction Log

This Model provides a record of an interaction with a content item. As such, it should encode the channel that the content was in, and the id of the content. Further, it may be required to encode arbitrary data in a JSON blob that is specific to the particular content type.

As a typical use case, a ContentInteractionLog object might be used to record an interaction with one instance of an exercise (i.e. one question, but possibly multiple attempts within the same session), or a single session of viewing a video.

Finally, these Logs will use MorangoDB to synchronize their data across devices.

Content Summary Log

This Model provides a summary of all interactions of a user with a content item. As such, it should encode the channel that the content was in, and the id of the content. Further, it may be required to encode arbitrary data in a JSON blob that is specific to the particular content type.

As a typical use case, a ContentSummaryLog object might be used to provide summary data about the state of completion of a particular exercise, video, or other content.

When a new InteractionLog is saved, the associated SummaryLog is updated at the same time. This means that the SummaryLog acts as an aggregation layer for the current state of progress for a particular piece of content.

To implement this, a content viewer app would define the aggregation function that summarizes interaction logs into the summary log. While this could happen in the frontend, it would probably be more efficient for this to happen on the server side.

Finally, these Logs will use MorangoDB to synchronize their data across devices - in the case where two summary logs from different devices conflict, then the aggregation logic would be applied across all interaction logs to create a consolidated summary log.

Content Rating Log

This Model provides a record of user feedback on content.

As a typical use case, a ContentRatingLog object might be used to record user feedback data about any content.

Finally, these Logs will use MorangoDB to synchronize their data across devices.

User Session Log

This Model provides a record of an user session in Kolibri. As such, it should encode the channels interacted with, the length of time engaged, and the pages visited.

As a typical use case, a UserSessionLog object might be used to record which pages a user visits, and how long the user is logged on for.

Finally, these Logs will use MorangoDB to synchronize their data across devices.

Implementation Details

Permissions

See Encoding Permission Rules.

Models

This app is intended to provide the core functionality for tracking user engagement with content and Kolibri in general. As such, it is intended to store details of user interactions with content, a summary of those interactions, interactions with the software in general, as well as user feedback on the content and the software.

class kolibri.logger.models.AttemptLog(*args, **kwargs)[source]

This model provides a summary of a user’s engagement within a particular interaction with an item/question in an assessment

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • item (CharField) – Item
  • start_timestamp (DateTimeTzField) – Start timestamp
  • end_timestamp (DateTimeTzField) – End timestamp
  • completion_timestamp (DateTimeTzField) – Completion timestamp
  • time_spent (FloatField) – (in seconds)
  • complete (BooleanField) – Complete
  • correct (FloatField) – Correct
  • hinted (BooleanField) – Hinted
  • answer (JSONField) – Answer
  • simple_answer (CharField) – Simple answer
  • interaction_history (JSONField) – Interaction history
  • user_id (ForeignKey to ~) – User
  • masterylog_id (ForeignKey to ~) – Masterylog
  • sessionlog_id (ForeignKey to ~) – Sessionlog
exception DoesNotExist
exception MultipleObjectsReturned
infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.BaseAttemptLog(*args, **kwargs)[source]

This is an abstract model that provides a summary of a user’s engagement within a particular interaction with an item/question in an assessment

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • item (CharField) – Item
  • start_timestamp (DateTimeTzField) – Start timestamp
  • end_timestamp (DateTimeTzField) – End timestamp
  • completion_timestamp (DateTimeTzField) – Completion timestamp
  • time_spent (FloatField) – (in seconds)
  • complete (BooleanField) – Complete
  • correct (FloatField) – Correct
  • hinted (BooleanField) – Hinted
  • answer (JSONField) – Answer
  • simple_answer (CharField) – Simple answer
  • interaction_history (JSONField) – Interaction history
  • user_id (ForeignKey to ~) – User
class kolibri.logger.models.BaseLogModel(*args, **kwargs)[source]
Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.BaseLogQuerySet(model=None, query=None, using=None, hints=None)[source]
filter_by_content_ids(content_ids, content_id_lookup='content_id')[source]

Filter a set of logs by content_id, using content_ids from the provided list or queryset.

filter_by_topic(topic, content_id_lookup='content_id')[source]

Filter a set of logs by content_id, using content_ids from all descendants of specified topic.

class kolibri.logger.models.ContentSessionLog(*args, **kwargs)[source]

This model provides a record of interactions with a content item within a single visit to that content page.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • user_id (ForeignKey to ~) – User
  • content_id (UUIDField) – Content id
  • channel_id (UUIDField) – Channel id
  • start_timestamp (DateTimeTzField) – Start timestamp
  • end_timestamp (DateTimeTzField) – End timestamp
  • time_spent (FloatField) – (in seconds)
  • progress (FloatField) – Progress
  • kind (CharField) – Kind
  • extra_fields (JSONField) – Extra fields
exception DoesNotExist
exception MultipleObjectsReturned
class kolibri.logger.models.ContentSummaryLog(*args, **kwargs)[source]

This model provides a summary of all interactions a user has had with a content item.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • user_id (ForeignKey to ~) – User
  • content_id (UUIDField) – Content id
  • channel_id (UUIDField) – Channel id
  • start_timestamp (DateTimeTzField) – Start timestamp
  • end_timestamp (DateTimeTzField) – End timestamp
  • completion_timestamp (DateTimeTzField) – Completion timestamp
  • time_spent (FloatField) – (in seconds)
  • progress (FloatField) – Progress
  • kind (CharField) – Kind
  • extra_fields (JSONField) – Extra fields
exception DoesNotExist
exception MultipleObjectsReturned
calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

class kolibri.logger.models.ExamAttemptLog(*args, **kwargs)[source]

This model provides a summary of a user’s engagement within a particular interaction with an item/question in an exam

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • item (CharField) – Item
  • start_timestamp (DateTimeTzField) – Start timestamp
  • end_timestamp (DateTimeTzField) – End timestamp
  • completion_timestamp (DateTimeTzField) – Completion timestamp
  • time_spent (FloatField) – (in seconds)
  • complete (BooleanField) – Complete
  • correct (FloatField) – Correct
  • hinted (BooleanField) – Hinted
  • answer (JSONField) – Answer
  • simple_answer (CharField) – Simple answer
  • interaction_history (JSONField) – Interaction history
  • user_id (ForeignKey to ~) – User
  • examlog_id (ForeignKey to ~) – Examlog
  • content_id (UUIDField) – Content id
  • channel_id (UUIDField) – Channel id
exception DoesNotExist
exception MultipleObjectsReturned
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.ExamLog(*args, **kwargs)[source]

This model provides a summary of a user’s interaction with a particular exam, and serves as an aggregation point for individual attempts on that exam.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • exam_id (ForeignKey to ~) – Exam
  • user_id (ForeignKey to ~) – User
  • closed (BooleanField) – Closed
  • completion_timestamp (DateTimeTzField) – Completion timestamp
exception DoesNotExist
exception MultipleObjectsReturned
calculate_partition()[source]

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

class kolibri.logger.models.MasteryLog(*args, **kwargs)[source]

This model provides a summary of a user’s engagement with an assessment within a mastery level

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • user_id (ForeignKey to ~) – User
  • summarylog_id (ForeignKey to ~) – Summarylog
  • mastery_criterion (JSONField) – Mastery criterion
  • start_timestamp (DateTimeTzField) – Start timestamp
  • end_timestamp (DateTimeTzField) – End timestamp
  • completion_timestamp (DateTimeTzField) – Completion timestamp
  • mastery_level (IntegerField) – Mastery level
  • complete (BooleanField) – Complete
exception DoesNotExist
exception MultipleObjectsReturned
calculate_source_id()[source]

Should return a string that uniquely defines the model instance or None for a random uuid.

infer_dataset(*args, **kwargs)[source]

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.UserSessionLog(*args, **kwargs)[source]

This model provides a record of a user session in Kolibri.

Parameters:
  • id (UUIDField) – Id
  • _morango_dirty_bit (BooleanField) – morango dirty bit
  • _morango_source_id (CharField) – morango source id
  • _morango_partition (CharField) – morango partition
  • dataset_id (ForeignKey to ~) – Dataset
  • user_id (ForeignKey to ~) – User
  • channels (TextField) – Channels
  • start_timestamp (DateTimeTzField) – Start timestamp
  • last_interaction_timestamp (DateTimeTzField) – Last interaction timestamp
  • pages (TextField) – Pages
exception DoesNotExist
exception MultipleObjectsReturned
classmethod update_log(user)[source]

Update the current UserSessionLog for a particular user.

Kolibri plugin architecture

The behavior of Kolibri can be extended using plugins. The following is a guide to developing plugins.

Enabling and disabling plugins

Non-core plugins can be enabled or disabled using the kolibri plugin command. See ../user/cli.

How plugins work

From a user’s perspective, plugins are enabled and disabled through the command line interface or through a UI. Users can also configure a plugin’s behavior through the main Kolibri interface.

Note

We have not yet written a configuration API, for now just make sure configuration-related variables are kept in a central location of your plugin.

It’s up to the plugin to provide configuration Form classes and register them.

We should aim for a configuration style in which data can be pre-seeded, dumped and exported easily.

From a developer’s perspective, plugins are Django applications listed in INSTALLED_APPS and are initialized once when the server starts, mean at the load time of the django project, i.e. Kolibri.

Loading a plugin

In general, a plugin should never modify internals of Kolibri or other plugins without using the hooks API or normal conventional Django scenarios.

Note

Each app in INSTALLED_APPS is searched for the special kolibri_plugin module.

Everything that a plugin does is expected to be defined through <myapp>/kolibri_plugin.py.

Kolibri Hooks API
What are hooks

Hooks are classes that define something that happens at one or more places where the hook is looked for and applied. It means that you can “hook into a component” in Kolibri and have it do a predefined and parameterized thing. For instance, Kolibri could ask all its plugins who wants to add something to the user settings panel, and its then up to the plugins to inherit from that specific hook and feed back the parameters that the hook definition expects.

The consequences of a hook being applied can happen anywhere in Kolibri. Each hook is defined through a class inheriting from KolibriHook. But how the inheritor of that class deals with plugins using it, is entirely up to each specific implementation and can be applied in templates, views, middleware - basically everywhere!

That’s why you should consult the class definition and documentation of the hook you are adding plugin functionality with.

We have two different types of hooks:

Abstract hooks
Are definitions of hooks that are implemented by implementing hooks.
Registered hooks
Are concrete hooks that inherit from abstract hooks, thus embodying the definitions of the abstract hook into a specific case.
So what’s “a hook”?
Simply referring to “a hook” is okay, it can be ambiguous on purpose. For instance, in the example, we talk about “a navigation hook”. So we both mean the abstract definition of the navigation hook and everything that is registered for the navigation.
Where can I find hooks?

All Kolibri core applications and plugins alike should by convention define their abstract hooks inside <myapp>/hooks.py. Thus, to see which hooks a Kolibri component exposes, you can refer to its hooks module.

Note

Defining abstract hooks in <myapp>/hooks.py isn’t mandatory, but loading a concrete hook in <myapp>/kolibri_plugin.py is.

Warning

Do not define abstract and registered hooks in the same module. Or to put it in other words: Never put registered hooks in <myapp>/hooks.py. The non-abstract hooks should not be loaded unintentionally in case your application is not loaded but only used to import an abstract definition by an external component!

In which order are hooks used/applied?

This is entirely up to the registering class. By default, hooks are applied in the same order that the registered hook gets registered! This most likely means the order in which kolibri_plugin is loaded => the order in which the app is listed in INSTALLED_APPS

An example of a plugin using a hook

Note

The example shows a NavigationHook which is simplified for the sake of readability. The actual implementation in Kolibri will defer.

Example implementation

Here is an example of how to use a hook in myplugin.kolibri_plugin.py:

from django.db.models

# This is where the actual abstract hook is defined
from kolibri.core.hooks import NavigationHook

# By inheriting NavigationHook, we tell that we are going to want our
# plugin to be part of the hook's activities with the specified attributes.
# We only define one navigation item, but defining another one is as simple
# as adding another class definition.
class MyPluginNavigationItem(NavigationHook):

    label = _("My Plugin")
    url = reverse_lazy("kolibri:my_plugin:index")

And here is the definition of that hook in kolibri.core.hooks:

from kolibri.plugins.hooks import KolibriHook


class NavigationHook(KolibriHook):
    """
    Extend this hook to define a new navigation item
    """

    #: A string label for the menu item
    label = "Untitled"

    #: A string or lazy proxy for the url
    url = "/"

    @classmethod
    def get_menu(cls):
        menu = {}
        for hook in self.registered_hooks:
            menu[hook.label] = url
        return menu

    class Meta:

        abstract = True
Usage of the hook

Inside our templates, we load a template tag from navigation_tags, and this template tag definition looks like this:

from kolibri.core.hooks import NavigationHook

@register.assignment_tag()
def kolibri_main_navigation():

    for item in NavigationHook().get_menu():
        yield item

{% load kolibri_tags %}

<ul>
{% for menu_item in kolibri_main_navigation %}
    <li><a href="{{ menu_item.url }}">{{ menu_item.label }}</a></li>
{% endfor %}
</ul>

Warning

Do not load registered hook classes outside of a plugin’s kolibri_plugin. Either define them there directly or import the modules that define them. Hook classes should all be seen at load time, and placing that logic in kolibri_plugin guarantees that things are registered correctly.

Other stuff you can do with plugins

Plugins can implement Javascript code as a Kolibri module that can be used in the frontend as a plugin to the core Kolibri Javascript code. Each of these Javascript plugins are defined in the kolibri_plugin.py file by subclassing the KolibriFrontEndPluginBase class to define each frontend Kolibri module. This defines the base Javascript file that defines the Kolibri module. In addition, this Plugin object within the app will automatically add these Kolibri modules to an internal frontend asset registry for loading in the front end. For more information on developing frontend code for Kolibri please see frontend.

Plugins can be standalone Django apps in their own right, meaning they can define templates, models, new urls, and views just like any other app. However the API for all of this hasn’t yet been determined… Coming soon!

Core plugin example

View the source to learn more!

class kolibri.core.kolibri_plugin.KolibriCore[source]

The most minimal plugin possible. Because it’s in the core, it doesn’t define enable or disable. Those methods should never be called for this plugin.

Builds

Distribution build pipeline

The Kolibri Package build pipeline looks like this:

                       Git master branch
                               |
                               |
                              / \
                             /   \
Python dist, online dependencies  \
   `python setup.py bdist_wheel`   \
               /                    \
              /                Python dist, bundled dependencies
       Upload to PyPi        `python setup.py bdist_wheel --static`
      Installable with                 \
    `pip install kolibri`               \
                                   Upload to PyPi
                                   Installable with
                             `pip install kolibri-static`
                               /            |          \
                              /             |           \
                        Windows            OSX        Debian
                       installer         installer   installer

Make targets

To build both the slim Kolibri and the one with bundled dependencies, simply run make dist. The .whl files will now be available in dist/*whl and you can install them with pip install dist/filename.whl.

Front-end build pipeline

Asset pipelining is done using Webpack - this allows the use of require to import modules - as such all written code should be highly modular, individual files should be responsible for exporting a single function or object.

There are two distinct entities that control this behaviour - a Kolibri Hook on the Python side, which manages the registration of the frontend code within Django (and also facilitates building of that code into compiled assets with Webpack) and a Kolibri Module (a subclass of KolibriModule) on the JavaScript side (see frontend).

Kolibri has a system for synchronously and asynchronously loading these bundled JavaScript modules which is mediated by a small core JavaScript app, kolibriGlobal. Kolibri Modules define to which events they subscribe, and asynchronously registered Kolibri Modules are loaded by kolibriGlobal only when those events are triggered. For example if the Video Viewer’s Kolibri Module subscribes to the content_loaded:video event, then when that event is triggered on kolibriGlobal it will asynchronously load the Video Viewer module and re-trigger the content_loaded:video event on the object the module returns.

Synchronous and asynchronous loading is defined by the template tag used to import the JavaScript for the Kolibri Module into the Django template. Synchronous loading merely inserts the JavaScript and CSS for the Kolibri Module directly into the Django template, meaning it is executed at page load.

This can be achieved in two ways using tags defined in kolibri/core/webpack/templatetags/webpack_tags.py.

The first way is simply by using the webpack_asset template tag.

The second way is if a Kolibri Module needs to load in the template defined by another plugin or a core part of Kolibri, a template tag and hook can be defined to register that Kolibri Module’s assets to be loaded on that page. An example of this is found in the base.html template using the webpack_base_assets tag.

This relies on the following function to collect all registered Kolibri Modules and load them synchronously: kolibri.core.webpack.utils.webpack_asset_render

Asynchronous loading can also, analogously, be done in two ways. Asynchronous loading registers a Kolibri Module against kolibriGlobal on the frontend at page load, but does not load, or execute any of the code until the events that the Kolibri Module specifies are triggered. When these are triggered, the kolibriGlobal will load the Kolibri Module and pass on any callbacks once it has initialized. Asynchronous loading can be done either explicitly with a template tag that directly imports a single Kolibri Module using webpack_base_async_assets.

Upgrading

Warning

These instructions are under development

Upgrade paths

Kolibri can be automatically upgraded forwards. For instance, you can upgrade from 0.1->0.2 and 0.1->0.7. We test all upgrade paths, but we also caution that the more versions that you skip, the higher the risks will be that something isn’t working as expected.

That’s why we also support Downgrading.

Every time Kolibri is upgraded, it will automatically migrate your database and create a backup before doing so.

Note

Always upgrade as often as possible. If you are responsible for deployments at different sites, you should consider a strategy for keeping software and contents updated.

Downgrading

To downgrade you need to do two steps:

  1. If you have been using the latest version and want to store data, make sure to create a backup before continuing: kalite manage dbbackup
  2. Install the older version on top of the new version using the same installation type.
  3. Restore the latest Database backup.

When you upgrade Kolibri, the database is changed to match the latest version of Kolibri, however these changes cannot be unmade. That’s why you need to restore the database from a backup.

Database backup

While upgrading, Kolibri will automatically generate a backup of the database before making any changes. This guarantees that in case the upgrade causes problems, you can downgrade and restore the backup.

Backups

Kolibri stores database backups in ~/.kolibri/backups. The dump files created contain SQL statements to be run by SQLite3. You can re-instate a dump by using the special dbrestore command.

Restoring from backup

Warning

Restoring from backup will overwrite the current database, so store a backup in case you have data you want to preserve!

To restore from the latest available backup, run the following from command line:

$ kolibri manage dbrestore --latest

To restore from a specific backup file:

$ kolibri manage dbrestore /path/to/db-backup.dump