Developer Guide

First of all, please read our gettting started guide. to learn about setting up your development environment, a nice development workflow and our technology stack.

If you are looking for help installing, configuring and using Kolibri, please refer to the Kolibri User Guide.

Kolibri

What is Kolibri?

Kolibri is a Learning Management System / Learning App designed to run on low-power devices, targeting the needs of learners and teachers in contexts with limited infrastructure. A user can install Kolibri and serve the app on a local network, without an internet connection. Kolibri installations can be linked to one another, so that user data and content can be shared. Users can create content for Kolibri and share it when there is network access to another Kolibri installation or the internet.

At its core, Kolibri is about serving educational content. A typical user (called a Learner) will log in to Kolibri to consume educational content (videos, documents, other multimedia) and test their understanding of the content by completing exercises and quizzes, with immediate feedback. A user’s activity will be tracked to offer individualized insight (like “next lesson” recommendations), and to allow user data to be synced across different installations – thus a Kolibri learner can use his or her credentials on any linked Kolibri installation, for instance on different devices at a school.

See our website for more information.

How can I use it?

An initial version of Kolibri is now available for download!

How can I contribute?

Thanks for your interest! Please see the contributing section of our online developer documentation.

Table of contents

Start here

Getting started

First of all, thank you for your interest in contributing to Kolibri! The project was founded by volunteers dedicated to helping make educational materials more accessible to those in need, and every contribution makes a difference. The instructions below should get you up and running the code in no time!

Setting up Kolibri for development

Most of the steps below require entering commands into your Terminal (Linux, Mac) or command prompt (cmd.exe on Windows) that you will learn how to use and become more comfortable with.

Tip

In case you run into any problems during these steps, searching online is usually the fastest way out: whatever error you are seeing, chances are good that somebody already had it in the past and posted a solution somewhere… ;)

Git & GitHub

1. Install and set-up Git on your computer. Try this tutorial if you need more practice with Git!
2. Sign up and configure your GitHub account if you don’t have one already.
3. Fork the main Kolibri repository. This will make it easier to submit pull requests. Read more details about forking from GitHub.

Install Environment Dependencies

1. Install Python if you are on Windows, on Linux and OSX Python is preinstalled (recommended versions 2.7+ or 3.4+).

2. Install pip package installer.

3. Install Node (version 6 is required).

4. Install Yarn according the instructions specific for your OS.

   Note

   • On Ubuntu install Node.js via nvm to avoid build issues.
   • On a Mac, you may want to consider using the Homebrew package manager.

Ready for the fun part in the Terminal? Here we go!

Checking out the code

1. Make sure you registered your SSH keys on GitHub.

2. Clone your Kolibri fork to your local computer. In the following commands replace $USERNAME with your own GitHub username:

   # using SSH
   git clone git@github.com:$USERNAME/kolibri.git
   # using HTTPS
   git clone https://github.com/$USERNAME/kolibri.git
   

3. Enable syncing your local repository with upstream, which refers to the Kolibri source from where you cloned your fork. That way you can keep it updated with the changes from the rest of Kolibri team contributors:

cd kolibri  # Change into the newly cloned directory
git remote add upstream git@github.com:learningequality/kolibri.git  # Add the upstream
git fetch upstream # Check if there are changes upstream
git checkout develop

Warning

develop is the active development branch - do not target the master branch.

Virtual environment

It is best practice to use Python virtual environment to isolate the dependencies of your Python projects from each other. This also allows you to avoid using sudo with pip, which is not recommended.

You can learn more about using virtualenv, or follow these basic instructions:

Initial setup, performed once:

$ sudo pip install virtualenv  # install virtualenv globally
$ mkdir ~/.venvs               # create a common directory for multiple virtual environments
$ virtualenv ~/.venvs/kolibri  # create a new virtualenv for Kolibri dependencies

Note

We create the virtualenv outside of the Kolibri project folder. You can choose another location than ~/.venvs/kolibri if desired.

To activate the virtualenv in a standard Bash shell:

$ source ~/.venvs/kolibri/bin/activate  # activate the venv

Now, any commands run with pip will target your virtualenv rather than the global Python installation.

To deactivate the virtualenv, run the command below. Note, you’ll want to leave it activated for the remainder of project setup!

$ deactivate

Tip

• Users of Windows and other shells such as Fish should read the guide for instructions on activating.
• If you set the PIP_REQUIRE_VIRTUALENV environment variable to true, pip will only install packages when a virtualenv is active. This can help prevent mistakes.
• Bash users might also consider using virtualenvwrapper, which simplifies the process somewhat.

Install Project Dependencies

Note

Make sure your virtualenv is active!

To install Kolibri project-specific dependencies make sure you’re in the kolibri directory and run:

# Python requirements
(kolibri)$ pip install -r requirements.txt
(kolibri)$ pip install -r requirements/dev.txt

# Kolibri Python package in 'editable' mode, so your installation points to your git checkout:
(kolibri)$ pip install -e .

# Javascript dependencies
(kolibri)$ yarn install

Tip

• We’ve adopted this concatenated version with added cleanup: pip install -r requirements/dev.txt --upgrade && make clean && pip install -e . && yarn install.
• In case you get webpack compilation error with Node modules build failures, add the flag --force at the end, to ensure binaries get installed.

Running Kolibri server

Development server

To start up the development server and build the client-side dependencies, use the following command:

(kolibri)$ yarn run devserver

If this does not work, you should run the commands it is invoking in two separate terminal windows, the first runs the django development server:

(kolibri)$ kolibri --debug manage devserver --settings=kolibri.deployment.default.settings.dev

The second runs the webpack build process for frontend assets in ‘watch’ mode, meaning they will be automatically rebuilt if you modify them.

(kolibri)$ yarn run watch

Wait for the build process to complete. This takes a while the first time, will complete faster as you make edits and the assets are automatically re-built.

Now you should be able to access the server at http://127.0.0.1:8000/.

Tip

If you need to make the development server available through the LAN, you need to do a production build of the assets, so use the following command:

(kolibri)$ yarn run build
(kolibri)$ kolibri --debug manage devserver -- 0.0.0.0:8000

Now you can simply use your server’s IP from another device in the local network through the port 8000, for example http://192.168.1.38:8000/.

Tip

If get an error similar to Node Sass could not find a binding for your current environment try running:

(kolibri)$ npm rebuild node-sass

Running the Production Server

In production, content is served through CherryPy. Static assets must be pre-built:

yarn run build
kolibri start

Now you should be able to access the server at http://127.0.0.1:8080/.

Additional Recommended Setup

If you’re planning on contributing code to the project, there are a few additional steps you should consider taking.

Editor Config

We have a project-level .editorconfig file to help you configure your text editor or IDE to use our internal conventions.

Check your editor to see if it supports EditorConfig out-of-the-box, or if a plugin is available.

Front-end Dev Tools

If you’re working with front-end Vue.js and use Google Chrome Dev Tools, you may find the Vue.js devtools helpful

DB Setup

You can initialize the server using:

kolibri manage migrate

Pre-Commit Install

We use pre-commit to help ensure consistent, clean code. The pip package should already be installed from a prior setup step, but you need to install the git hooks using this command.

pre-commit install

Development workflows

Linting

Javascript linting is always run when you run the dev server. In addition, all frontend assets that are bundled will be linted by our Travis CI builds. It is a good idea, therefore, to monitor for linting errors in the webpack build process, while the build will complete in watch mode, it will issue warnings to the terminal.

Code Testing

First, install some additional dependencies related to running tests:

pip install -r requirements/test.txt

Kolibri comes with a Python test suite based on py.test. To run tests in your current environment:

pytest  # alternatively, "make test" does the same

You can also use tox to setup a clean and disposable environment:

tox -e py3.4  # Runs tests with Python 3.4

To run Python tests for all environments, lint and documentation tests, use simply tox. This simulates what our CI also does.

To run Python linting tests (pep8 and static code analysis), use tox -e lint or make lint.

Note that tox reuses its environment when it is run again. If you add anything to the requirements, you will want to either delete the .tox directory, or run tox with the -r argument to recreate the environment.

We strive for 100% code coverage in Kolibri. When you open a Pull Request, code coverage (and your impact on coverage) will be reported. To test code coverage locally, so that you can work to improve it, you can run the following:

tox -e py3.4
coverage html

Then, open the generated ./htmlcov/index.html file in your browser.

Kolibri comes with a Javascript test suite based on mocha. To run all tests:

yarn test

This includes tests of the bundling functions that are used in creating front end assets. To do continuous unit testing for code, and jshint running:

yarn run test-karma:watch

To run specific tests only, you can add the filepath of the file. To further filter either by TestClass name or test method name, you can add -k followed by a string to filter classes or methods by. For example, to only run a test named test_admin_can_delete_membership in kolibri/auth/test/test_permissions.py:

pytest kolibri/auth/test/test_permissions -k test_admin_can_delete_membership

To only run the whole class named MembershipPermissionsTestCase in kolibri/auth/test/test_permissions.py:

pytest kolibri/auth/test/test_permissions -k MembershipPermissionsTestCase

For more advanced usage, logical operators can also be used in wrapped strings, for example, the following will run only one test, named test_admin_can_delete_membership in the MembershipPermissionsTestCase class in kolibri/auth/test/test_permissions.py:

pytest kolibri/auth/test/test_permissions -k "MembershipPermissionsTestCase and test_admin_can_delete_membership"

Updating Documentation

First, install some additional dependencies related to building documentation output:

pip install -r requirements/docs.txt
pip install -r requirements/build.txt

To make changes to documentation, edit the rst files in the kolibri/docs directory and then run:

make docs

You can also run the auto-build for faster editing from the docs directory:

cd docs
sphinx-autobuild --port 8888 . _build

Manual Testing

All changes should be thoroughly tested and vetted before being merged in. Our primary considerations are:

• Performance
• Accessibility
• Compatibility
• Localization
• Consistency

For more information, see the next section on Manual testing & QA.

Submitting a Pull Request

The most common situation is working off of develop branch so we’ll take it as an example:

$ git checkout upstream/develop
$ git checkout -b name-of-your-bugfix-or-feature

After making changes to the code, commit and push them to a branch on your fork:

$ git add -A  # Add all changed and new files to the commit
$ git commit -m "Write here the commit message"
$ git push origin name-of-your-bugfix-or-feature

Go to Kolibri GitHub page, and if you are logged-in you will see the link to compare your branch and and create the new pull request. Please fill in all the applicable sections in the PR template and DELETE unecessary headings. Another member of the team will review your code, and either ask for updates on your part or merge your PR to Kolibri codebase. Until the PR is merged you can push new commits to your branch and add updates to it.

Learn more about our Release process

Contributing

Ways to contribute

Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.

Talk to us

• Get support in our Community Forums.
• Email us at info@learningequality.org
• Sign up to receive developer announcements in Google groups
• Visit the #kolibri room on Freenode IRC

Translate

Help us translate the application on Crowdin.

Give Feedback

Report issues on github. Please search the existing issues first to see if it’s already been reported.

If you are reporting a bug, please include:

• Your operating system name and version.
• Any details about your local setup that might be helpful in troubleshooting.
• Detailed steps to reproduce the bug.

If you are proposing a new feature or giving feedback on an existing feature:

• Explain in detail what you’re trying to do and why the existing system isn’t working for you
• Keep the scope as narrow as possible, to make it easier to understand the specific problem you’re trying to address

Write Code

Look through the GitHub issues for issues you’d like to address. You can check our upcoming milestones to look for high-priority items.

Then, visit Getting started to begin contributing.

Note that since Kolibri is still in development, the APIs are subject to change, and a lot of code is still in flux.

Write Documentation

Kolibri could always use more documentation, whether as part of the official Kolibri docs, in docstrings, or even on the web in blog posts, articles, and such.

Code of conduct

Code of Conduct

1. Purpose

A primary goal of Kolibri and KA Lite is to be inclusive to the largest number of contributors, with the most varied and diverse backgrounds possible. As such, we are committed to providing a friendly, safe and welcoming environment for all, regardless of gender, sexual orientation, ability, ethnicity, socioeconomic status, and religion (or lack thereof).

This code of conduct outlines our expectations for all those who participate in our community, as well as the consequences for unacceptable behavior.

We invite all those who participate in Kolibri or KA Lite to help us create safe and positive experiences for everyone.

2. Open Source Citizenship

A supplemental goal of this Code of Conduct is to increase open source citizenship by encouraging participants to recognize and strengthen the relationships between our actions and their effects on our community.

Communities mirror the societies in which they exist and positive action is essential to counteract the many forms of inequality and abuses of power that exist in society.

If you see someone who is making an extra effort to ensure our community is welcoming, friendly, and encourages all participants to contribute to the fullest extent, we also want to know!

3. Expected Behavior

The following behaviors are expected and requested of all community members:

• Participate in an authentic and active way. In doing so, you contribute to the health and longevity of this community.
• Exercise consideration and respect in your speech and actions.
• Attempt collaboration before conflict.
• Refrain from demeaning, discriminatory, or harassing behavior and speech.
• Be mindful of your surroundings and of your fellow participants. Alert community leaders if you notice a dangerous situation, someone in distress, or violations of this Code of Conduct, even if they seem inconsequential.
• Remember that community event venues may be shared with members of the public; please be respectful to all patrons of these locations.

4. Unacceptable Behavior

The following behaviors are considered harassment and are unacceptable within our community:

• Violence, threats of violence or violent language directed against another person.
• Sexist, racist, homophobic, transphobic, ableist or otherwise discriminatory jokes and language.
• Posting or displaying sexually explicit or violent material.
• Posting or threatening to post other people’s personally identifying information (“doxing”).
• Personal insults, particularly those related to gender, sexual orientation, race, religion, or disability.
• Inappropriate photography or recording.
• Inappropriate physical contact. You should have someone’s consent before touching them.
• Unwelcome sexual attention. This includes, sexualized comments or jokes; inappropriate touching, groping, and unwelcomed sexual advances.
• Deliberate intimidation, stalking or following (online or in person).
• Advocating for, or encouraging, any of the above behavior.
• Sustained disruption of community events, including talks and presentations.

5. Consequences of Unacceptable Behavior

Unacceptable behavior from any community member, including sponsors and those with decision-making authority, will not be tolerated.

Anyone asked to stop unacceptable behavior is expected to comply immediately.

If a community member engages in unacceptable behavior, the community organizers may take any action they deem appropriate, up to and including a temporary ban or permanent expulsion from the community without warning (and without refund in the case of a paid event).

6. Reporting Guidelines

If you are subject to or witness unacceptable behavior, or have any other concerns, please notify a community organizer as soon as possible. codeofconduct@learningequality.org.

Reporting Guidelines

Additionally, community organizers are available to help community members engage with local law enforcement or to otherwise help those experiencing unacceptable behavior feel safe. In the context of in-person events, organizers will also provide escorts as desired by the person experiencing distress.

7. Addressing Grievances

If you feel you have been falsely or unfairly accused of violating this Code of Conduct, you should notify Learning Equality with a concise description of your grievance. Your grievance will be handled in accordance with our existing governing policies.

Enforcement Manual

8. Scope

We expect all community participants (contributors, paid or otherwise; sponsors; and other guests) to abide by this Code of Conduct in all community venues–online and in-person–as well as in all one-on-one communications pertaining to community business.

This code of conduct and its related procedures also applies to unacceptable behavior occurring outside the scope of community activities when such behavior has the potential to adversely affect the safety and well-being of community members.

9. Contact info

The Code of Conduct team consists of:

• Benjamin Bach (benjamin@learningequality.org)
• Radina Matic (radina@learningequality.org)
• Richard Tibbles (richard@learningequality.org)

Please write: codeofconduct@learningequality.org

10. License and attribution

This Code of Conduct is distributed under a Creative Commons Attribution-ShareAlike license.

Portions of text derived from the Django Code of Conduct and the Geek Feminism Anti-Harassment Policy.

Retrieved on November 22, 2016 from http://citizencodeofconduct.org/

Reporting Guidelines

If you believe someone is violating the code of conduct we ask that you report it to the Learning Equality by emailing codeofconduct@learningequality.org. All reports will be kept confidential. In some cases we may determine that a public statement will need to be made. If that’s the case, the identities of all victims and reporters will remain confidential unless those individuals instruct us otherwise.

If you believe anyone is in physical danger, please notify appropriate law enforcement first. If you are unsure what law enforcement agency is appropriate, please include this in your report and we will attempt to notify them.

If you are unsure whether the incident is a violation, or whether the space where it happened is covered by this Code of Conduct, we encourage you to still report it. We would much rather have a few extra reports where we decide to take no action, rather than miss a report of an actual violation. We do not look negatively on you if we find the incident is not a violation. And knowing about incidents that are not violations, or happen outside our spaces, can also help us to improve the Code of Conduct or the processes surrounding it.

In your report please include:

1. Your contact info (so we can get in touch with you if we need to follow up)
2. Names (real, nicknames, or pseudonyms) of any individuals involved. If there were other witnesses besides you, please try to include them as well.
3. When and where the incident occurred. Please be as specific as possible.
4. Your account of what occurred. If there is a publicly available record (e.g. a mailing list archive or a public Slack logger) please include a link.
5. Any extra context you believe existed for the incident.
6. If you believe this incident is ongoing.
7. Any other information you believe we should have.

What happens after you file a report?

You will receive an email from the Code of Conduct committee acknowledging receipt within 48 hours (we aim to be quicker than that).

The committee will immediately meet to review the incident and determine:

1. What happened.
2. Whether this event constitutes a code of conduct violation.
3. Who the bad actor was.
4. Whether this is an ongoing situation, or if there is a threat to anyone’s physical safety.

If this is determined to be an ongoing incident or a threat to physical safety, the committee’s immediate priority will be to protect everyone involved. This means we may delay an “official” response until we believe that the situation has ended and that everyone is physically safe.

Once the committee has a complete account of the events they will make a decision as to how to response. Responses may include:

• Nothing (if we determine no violation occurred).
• A private reprimand from the committee to the individual(s) involved.
• A public reprimand.
• An imposed vacation (i.e. asking someone to “take a week off” from a mailing list or Slack).
• A permanent or temporary ban from some or all communication spaces (mailing lists, Slack, etc.)
• A request for a public or private apology.

We’ll respond within one week to the person who filed the report with either a resolution or an explanation of why the situation is not yet resolved.

Once we’ve determined our final action, we’ll contact the original reporter to let them know what action (if any) we’ll be taking. We’ll take into account feedback from the reporter on the appropriateness of our response, but we don’t guarantee we’ll act on it.

Enforcement Manual

This is the enforcement manual followed by Learning Equality’s Code of Conduct Committee. It’s used when we respond to an issue to make sure we’re consistent and fair. It should be considered an internal document, but we’re publishing it publicly in the interests of transparency.

The Code of Conduct Committee

All responses to reports of conduct violations will be managed by a Code of Conduct Committee (“the committee”).

Learning Equality’s (LE’s) core team (“the core”) will establish this committee, comprised of at least three members.

How the committee will respond to reports

When a report is sent to the committee, a member will reply with a receipt to confirm that a process of reading your report has started.

See the reporting guidelines for details of what reports should contain. If a report doesn’t contain enough information, the committee will obtain all relevant data before acting. The committee is empowered to act on the LE’s behalf in contacting any individuals involved to get a more complete account of events.

The committee will then review the incident and determine, to the best of their ability:

• what happened
• whether this event constitutes a code of conduct violation
• who, if anyone, was the bad actor
• whether this is an ongoing situation, and there is a threat to anyone’s physical safety

This information will be collected in writing, and whenever possible the committee’s deliberations will be recorded and retained (i.e. Slack transcripts, email discussions, recorded voice conversations, etc).

The committee should aim to have a resolution agreed upon within one week. In the event that a resolution can’t be determined in that time, the committee will respond to the reporter(s) with an update and projected timeline for resolution.

Acting Unilaterally

If the act is ongoing or involves a threat to anyone’s safety (e.g. threats of violence), any committee member may act immediately (before reaching consensus) to end the situation. In ongoing situations, any member may at their discretion employ any of the tools available to the committee, including bans and blocks.

If the incident involves physical danger, any member of the committee may – and should – act unilaterally to protect safety. This can include contacting law enforcement (or other local personnel) and speaking on behalf of Learning Equality.

In situations where an individual committee member acts unilaterally, they must report their actions to the committee for review within 24 hours.

Resolutions

The committee must agree on a resolution by consensus. If the committee cannot reach consensus and deadlocks for over a week, the committee will turn the matter over to the board for resolution.

Possible responses may include:

• Taking no further action (if we determine no violation occurred).
• A private reprimand from the committee to the individual(s) involved. In this case, the committee will deliver that reprimand to the individual(s) over email, cc’ing the committee.
• A public reprimand. In this case, the committee will deliver that reprimand in the same venue that the violation occurred (i.e. in Slack for an Slack violation; email for an email violation, etc.). The committee may choose to publish this message elsewhere for posterity.
• An imposed vacation (i.e. asking someone to “take a week off” from a mailing list or Slack). The committee will communicate this “vacation” to the individual(s). They’ll be asked to take this vacation voluntarily, but if they don’t agree then a temporary ban may be imposed to enforce this vacation.
• A permanent or temporary ban from some or all Learning Equality spaces (mailing lists, Slack, etc.). The committee will maintain records of all such bans so that they may be reviewed in the future, extended to new Learning Equality fora, or otherwise maintained.
• A request for a public or private apology. The committee may, if it chooses, attach “strings” to this request: for example, the committee may ask a violator to apologize in order to retain his or her membership on a mailing list.

Once a resolution is agreed upon, but before it is enacted, the committee will contact the original reporter and any other affected parties and explain the proposed resolution. The committee will ask if this resolution is acceptable, and must note feedback for the record. However, the committee is not required to act on this feedback.

Finally, the committee will make a report for the core team.

The committee will never publicly discuss the issue; all public statements will be made by the core team.

Conflicts of Interest

In the event of any conflict of interest a committee member must immediately notify the other members, and recuse themselves if necessary.

Attribution

Reporting Guidelines and Enforcement Manual are both distributed under a Creative Commons Attribution-ShareAlike license.

Reporting Guidelines and Enforcement Manual are both derived from the Django’ Reporting Guidelines and Django’ Enforcement Manual

Changes made to the original doc: Instead of involving a board as DSF has, the core team at Learning Equality is considered. Instead of IRC, we refer to Slack. The Code of Conduct Committee does not have a single chair but acts as a group to make conflicts of interest easier, and to avoid problems in case of absence of the chair person. Instead of interchanging “working group” and “committee” notation, we replaced all occurrences of “working group” and “group” with “committee”.

Credits

Development Lead and Copyright Holder

• Learning Equality – info@learningequality.org

Community

Please feel free to add your name on this list if you do a PR!

• Benjamin Bach (benjaoming)
• Michael Gallaspy (MCGallaspy)
• Richard Tibbles (rtibbles)
• Jamie Alexandre (jamalex)
• David Cañas (dxcanas)
• Eli Dai (66eli77)
• Devon Rueckner (indirectlylit)
• Rafael Aguayo (ralphiee22)
• Christian Memije (christianmemije)
• Radina Matic (radinamatic)
• Mingqi Zhu (EmanekaT)
• Alan Chen (alanchenz)
• Yixuan Liu (yil039)
• Gerardo Soto (gcodeon)
• Paul Luna (luna215)
• Maureen Hernandez(mauhernandez)
• Lingyi Wang (lyw07)
• Richard Amodia (mrpau-richard)
• Eugene Oliveros (mrpau-eugene)
• Geoff Rich (geoffrey1218)

Tech stack overview

Kolibri is a web application built primarily using Python on the server-side and JavaScript on the client-side.

We use many run-time, development, and build-related technologies and tools, as outlined below.

Server

The server is a Django 1.9 application, and contains only pure-Python (2.7+) libraries dependencies at run-time.

The server is responsible for:

• Interfacing with the database (PostgreSQL) containing user, content, and language pack data
• Authentication and permission middleware
• Routing and handling of API calls, using the Django REST Framework
• Basic top-level URL routing between high-level sections of the application
• Serving basic HTML wrappers for the UI with data bootstrapped into the page
• Serving additional client assets such as fonts and images

TODO - how does Morango fit into this picture? Logging?

Client

The front-end user interface is built using HTML, the Stylus CSS-preprocessing language, and the ES2015 preset features of ES6 JavaScript.

The frontend targets IE9 and up, with an emphasis on tablet-size screens. We strive to use accessible, semantic HTML with support for screen readers, keyboard interaction, and right-to-left language support.

The client is responsible for:

• Compositing and rendering the UI using Vue.js components to build nested views
• Managing client-side state using Vuex
• Interacting with the server through HTTP requests

Additionally, loglevel is used for logging and normalize.css is included for browser style normalization.

Internationalization

We leverage the ICU Message syntax for formatting all user-facing text.

See Internationalization for more information.

Developer Docs

Documentation for Kolibri developers are formatted using reStructuredText and the output is generated using Sphinx. Most of the content is in the /docs directory, but some content is also extracted from Python source code and from files in the root directory. We use Read the Docs to host a public version of our documentation.

Additionally, information about the design and implementation of Kolibri might be found on Google Drive, Trello, Slack, InVision, mailing lists, office whiteboards, and lurking in the fragmented collective consciousness of our contributors.

Build Infrastructure

Client-side Resources

We use a combination of both Node.js and Python scripts to transform our source code as-written to the code that is run in a browser. This process involves webpack, plus a number of both custom and third-party extensions.

Preparation of client-side resources involves:

• ES6 to ES5
• Transforming Vue.js component files (*.vue) into JS and CSS
• Stylus to CSS
• Auto-prefixing CSS
• Bundling multiple JS dependencies into single files
• Minifying and compressing code
• Bundle resources such as fonts and images
• Generating source maps
• Providing mechanisms for decoupled “Kolibri plugins” to interact with each other and asynchronously load dependencies
• Linting to enforce code styles

Server Setup

The standard Django manage.py commands are used under-the-hood for database migration and user set-up.

TODO: is this accurate?

Installers and Packages

TODO: introduce stack (sdist, PyPi, Debian, Windows, etc)

see Distribution build pipeline

Continuous Integration

TODO: introduce stack (GitHub, CodeCov, Travis, buildkite, commit hooks)

Tests and Linting

We use a number of mechanisms to help encourage code quality and consistency. These checks enforce a subset of our Front-end code conventions.

• pre-commit is run locally on git commit and enforces some Python conventions
• We use EditorConfig to help developers set their editor preferences
• flake8 is also used to enforce Python conventions
• tox is used to run our test suites under a range of Python and Node environment versions
• sphinx-build -b linkcheck checks the validity of documentation links
• pytest runs our Python unit tests. We also leverage the Django test framework.
• In addition to building client assets, webpack runs linters on client-side code: ESLint for ES6 JavaScript, Stylint for Stylus, and HTMLHint for HTML and Vue.js components.
• Client-side code is tested using a stack of tools including Karma, Mocha, PhantomJS, Sinon, and rewire. TODO: Explain what each of these do
• codecov reports on the test coverage for Python and Node.js code. TODO - also client-side?

Helper Scripts

TODO: introduce stack (kolibri command, setup.py, makefiles, yarn commands, sphinx auto-build, etc)

References

Manual testing & QA

Accessibility (A11y) Testing

Inclusive design benefits all users, and we strive to make Kolibri accessible for all. Testing for accessibility can be challenging, but there are a few features you should check for before submitting your PR:

• Working keyboard navigation - everything that user can do with mouse or by touch must also work with the keyboard alone.
• Sufficient color contrast between foreground text/elements and the background.
• Meaningful text alternative for all non-decorative images, or an empty ALT attribute in case of decorative ones.
• Meaningful labels on ALL form or button elements.
• Page has one main heading (H1) and consecutive lower heading levels.

Here are a few tools that we use in testing for accessibility:

• WAVE Evaluation Tool - Firefox Add-on and Chrome extension.
• tota11y accessibility visualization toolkit - bookmarklet for Firefox and Chrome.
• Accessibility Developer Tools - Chrome extension.
• aXe Accessibility Engine - Firefox Add-on and Chrome extension.

There is a much longer list on our Kolibri Accessibility Tools Wiki page if you want to go deeper, but these four should be enough to help you avoid the most important accessibility pitfalls.

Cross-browser and OS Testing

It’s vital to ensure that our app works across a wide range of browsers and operating systems, particularly older versions of Windows and Android that are common on old and cheap devices.

In particular, we want to ensure that Kolibri runs on major browsers that match any of the following criteria:

• within the last two versions
• IE 9+ on Windows XP and up
• has at least 1% of global usage stats

Here are some useful options, in order of simplicity:

BrowserStack

BrowserStack is an incredibly useful tool for cross-browser and OS testing. In particular, it’s easy to install plugin which forwards localhost to a VM running on their servers, which in turn is displayed in your browser.

Amazon Workspaces

In some situations, simply having a browser is not enough. For example, a developer may need to test Windows-specific backend or installer code from another OS. In many situations, a virtual machine is appropriate - however these can be slow to download and run.

Amazon’s AWS Workspaces provides a faster alternative. They run Windows VMs in their cloud, and developers can RDP in.

Local Virtual Machines

Workspaces is very useful, but it has limitations: only a small range of OSes are available, and connectivity and provisioning are required.

An alternative is to run the guest operating system inside a virtual machine using e.g. VirtualBox. This also gives more developer flexibility, including e.g. shared directories between the guest and host systems. This tutorial shows how to test Kolibri in a VM.

Hardware

There are some situations where actual hardware is necessary to test the application. This is particularly true when virtualization might prohibit or impede testing features, such as lower-level driver interactions.

Responsiveness to Varying Screen Sizes

We want to ensure that the app looks and behaves reasonably across a wide range of typical screen sizes, from small tablets to large, HD monitors. It is highly recommended to constantly be testing functionality at a range of sizes.

Chrome and Firefox’s Developer Tools both have some excellent functionality to simulate arbitrary screen resolutions.

Slow Network Connection Speeds

It’s important to simulate end-users network conditions. This will help identify real-world performance issues that may not be apparent on local development machines.

Chrome’s Developer Tools have functionality to simulate a variety of network connections, including Edge, 3G, and even offline. An app can be loaded into multiple tabs, each with its own custom network connectivity profile. This will not affect traffic to other tabs.

Within the Chrome Dev Tools, navigate to the Network panel. Select a connection from the drop-down to apply network throttling and latency manipulation. When a Throttle is enabled the panel indicator will show a warning icon. This is to remind you that throttling is enabled when you are in other panels.

For Kolibri, our target audience’s network condition can be mimicked by setting connectivity to Regular 3G (100ms, 750kb/s, 250 kb/s).

Performance Testing with Django Debug Panel

We have built in support for Django Debug Panel (a Chrome extension that allows tracking of AJAX requests to Django).

To use this, ensure that you have development dependencies installed, and install the Django Debug Panel Chrome Extension. You can then run the development or production servers with the following environment variable set:

DJANGO_SETTINGS_MODULE=kolibri.deployment.default.settings.debug_panel

This will activate the debug panel, and will display in the Dev tools panel of Chrome. This panel will track all page loads and API requests. However, all data bootstrapping into the template will be disabled, as our data bootstrapping prevents the page load request from being profiled, and also does not profile the bootstrapped API requests.

Generating User Data

For manual testing, it is sometimes helpful to have generated user data, particularly for Coach and Admin facing functionality.

In order to do this, a management command is available:

kolibri manage generateuserdata

This will generate user data for the each currently existing channel on the system. Use the –help flag for options.

Release process

Branches and tags

• The master branch always has the latest stable code
• The develop branch is our current development branch
• Branches named like release-v1.2.x (for example) track all releases of the 1.2 release line. This may include multiple patch releases (like v1.2.0, v1.2.1, etc)
• Tags named like like v1.2.0-beta1 and v1.2.0 label specific releases

Note

At a high level, we follow the ‘Gitflow’ model. Some helpful references: Original reference, Atlassian

If a change needs to be introduced to an old release, target the oldest release branch that you want a bug fix introduced in. Then that will be merged into all later releases, including develop.

When we get close to releasing a new stable version/release of Kolibri, we generally branch develop into something like release-v0.1.x and tag it as a new beta. If you’re working on an issue targetted with that milestone, then you should target changes to that branch. Changes to those branches will later be pulled into develop again.

If you’re not sure which branch to target, ask the dev team!

Process

Update the Changelog

Update the Release Notes as necessary. In general we should try to keep the changelog up-to-date as PRs are merged in; however in practice the changelog usually needs to be cleaned up, fleshed out, and clarified.

Our changelogs should list:

• significant new features that were added
• significant categories of bug fixes or user-facing improvements
• significant behind-the-scenes technical improvements

Keep entries concise and consistent with the established writing style. The changelog should not include an entry for every PR or every issue closed. Reading the changelog should give a quick, high-level, semi-technical summary of what has changed.

Note that for older patch releases, the change should only be mentioned once: it is implied that fixes in older releases are propagated forward.

Additionally, we should also be adding the ‘changelog’ label to issues and pull requests on github. A more technical and granular overview of changes can be obtained by filtering by milestone and the ‘changelog’ label. Go through these issues and PRs, and ensure that the titles would be clear and meaningful.

Create a release branch

If this is a new major or minor release, you need to make a new branch as described above.

Pin installer versions

On Kolibri’s develop branch, we sometimes allow the installers to track the latest development versions on github. Before releasing Kolibri, we need to pin the Buildkite configuration to a tagged version of each installer.

Update any translation files

Make sure that the latest released translations are included for the appropriate release branch. Please see Crowdin workflow for details.

Finally, strings for any external Kolibri plugins (like kolibri-exercise-perseus-renderer) should also have been updated, a new release made, and the version updated in Kolibri. See the README of that repository for details.

Squash migrations

When possible, we like to utilize the Django migration squashing to simplify the migration path for new users (while simultaneously maintaining the migration path for old users). So far this has not been done, due to the existence of data migrations in our migration history. Once we have upgraded to Django 1.11, we will be able to mark these data migrations as elidable, and we will be able to better squash our history.

Ensure bugfixes from internal depencies have propagated

Some issues in Kolibri arise due to our integration of internally produced, but external to Kolibri, packages, such as kolibri-exercise-perseus-renderer, iceqube, and morango. If any of these kinds of dependencies have been updated to fix issues for this milestone, then the dependency version should have been updated.

Edit the VERSION file

Current practice is to bump kolibri.VERSION before tagging a release. You are allowed to have a newer version in kolibri.VERSION, but you are not allowed to add the tag before actually bumping kolibri.VERSION.

Select a release series number and initial version number:

$ SERIES=0.1.x
$ VER=0.1.0a

The form is:

       0.1.x
      /  |  \
     /   |   \
    /    |    \
major  minor   patch

Set the version in the release branch:

$ # edit VERSION in kolibri/__init__.py
$ git add kolibri/__init__.py
$ git commit -m "Bump version to $VER"

Set the version number in the develop branch if necessary.

Create a pull request on Github to get sign off for the release.

Checklist for sign off:

• [ ] Translation files have been updated
• [ ] Migrations have been squashed where possible
• [ ] Changelog has been updated
• [ ] LE Dependencies properly updated
• [ ] Tested Debian Installer
• [ ] Tested Windows Installer
• [ ] Tested PEX File

Tag the release

We always add git tags to a commit that makes it to a final or pre release. A tag is prefixed v and follows the Semver convention, for instance v1.2.3-alpha1.

Tag the release using github’s Releases feature.

Once a stable release is tagged, delete pre-releases (not the tags themselves) from github.

Copy the entries from the changelog into Github’s “Release notes”.

Warning

Always add tags in release branches. Otherwise, the tag chronology will break. Do not add tags in feature branches or in the master branch. You can add tags for pre-releases in develop, for releases that don’t yet have a release branch.

Warning

Tagging is known to break after rebasing, so in case you rebase a branch after tagging it, delete the tag and add it again. Basically, git describe --tags detects the closest tag, but after a rebase, its concept of distance is misguided.

Update version data

• Merge the release branch to current master if it’s the newest stable release.
• Change kolibri.VERSION to track the next development stage. Example: After releasing 1.0.0, change kolibri.VERSION to (1, 0, 1, 'alpha', 0) and commit to the release-v1.0.x branch.

Update milestone

• Close, if fixed, or change milestone of any issues on this release milestone.
• Close this milestone.

Release to PyPI

Select the version number and checkout the exact git tag:

$ VER=0.1.0
$ git checkout v$VER

Release with PyPI using the make command:

$ make release

Sign Windows installer

Use osslsigncode to sign the windows installer:

$ osslsigncode verify KolibriSetup-0.6.2.signed.exe

Sign and update the Debian PPA

[ TODO ]

Upload Windows installer and PEX file

Upload the PEX file and the signed windows installer to:

• /var/www/downloads/kolibri/vX.Y.Z/kolibri-vX.Y.Z.pex
• /var/www/downloads/kolibri/vX.Y.Z/kolibri-vX.Y.Z-windows-installer.exe

Make sure the files and parent directories are owned by the www-data user, e.g. by running:

sudo chown www-data:www-data [filename]

Update the online demo

Get kolibridemo.learningequality.org running the latest version:

• SSH into 192.237.248.135
• sudo su www-data
• cd ~/
• download new pex file and update the correct run...sh script

Then…:

sudo -i -u aron
killall python
run_all

Update learningequality.org

Update learningequality.org with the latest version number and release date. Currently, these two files need to be changed:

• fle_site/apps/main/templates/main/documentation.html
• fle_site/apps/main/templates/main/download.html

Also, update the LATEST_KOLIBRI_VERSION variable at this admin site.

Notifications

Tell the world!

[ TODO ]

• Announce release on dev list and newsletter if appropriate.
• For issues on this milestone that have been reported by the community, respond on the issues or other channels, notifying of the release that fixes this issues.

More on version numbers

Note

The content below is pulled from the docstring of the kolibri.utils.version module.

We follow semantic versioning 2.0.0 according to semver.org but for Python distributions and in the internal string representation in Python, you will find a PEP-440 flavor.

• 1.1.0 (Semver) = 1.1.0 (PEP-440).
• 1.0.0-alpha1 (Semver) = 1.0.0a1 (PEP-440).

Here’s how version numbers are generated:

• kolibri.__version__ is automatically set, runtime environments use it to decide the version of Kolibri as a string. This is especially something that PyPi and setuptools use.
• kolibri.VERSION is a tuple containing version information, it’s set in kolibri/__init__.py is automatically suffixed in pre-releases by a number of rules defined below. For a final release (not a pre-release), it will be used exactly as it appears.
• kolibri/VERSION is a file containing the exact version of Kolibri for a distributed environment (pre-releases only!)
• git describe --tags is a command run to fetch tag information from a git checkout with the Kolibri code. The information is used to validate the major components of kolibri.VERSION and to suffix the final version of prereleases. This information is stored permanently in kolibri/VERSION before shipping a pre-release by calling make writeversion during make dist etc.

Confused? Here’s a table:

Release type	kolibri.VERSION	kolibri/VERSION	Git data	Examples
Final	Canonical, only information used	N/A	N/A	0.1.0, 0.2.2, 0.2.post1
dev release (alpha0)	(1, 2, 3, ‘alpha’, 0), 0th alpha = a dev release! Never used as a canonical	Fallback	timestamp of latest commit + hash	0.4.0.dev020170605181124-f1234567
alpha1+	(1, 2, 3, ‘alpha’, 1)	Fallback	git describe --tags	Clean head: 1.2.3a1, Changes since tag: 1.2.3a1.dev123-f1234567
beta1+	(1, 2, 3, ‘alpha’, 1)	Fallback	git describe --tags	Clean head: 1.2.3b1, Changes since tag: 1.2.3b1.dev123-f1234567
rc1+ (release candidate)	(1, 2, 3, ‘alpha’, 1)	Fallback	git describe --tags	Clean head: 1.2.3rc1, Changes since tag: 1.2.3rc1.dev123-f1234567
beta0, rc0, post0, x.y.0	Not recommended, but if you use it, your release transforms into a X.Y.0b0.dev{suffix} release, which in most cases should be assigned to the preceding release type.	Fallback	timestamp of latest commit + hash	0.4.0b0.dev020170605181124-f1234567

Fallback: kolibri/VERSION is auto-generated with make writeversion during the build process. The file is read as a fallback when there’s no git data available in a pre-release (which is the case in an installed environment).

Release order example 1.2.3 release:

• VERSION = (1, 2, 3, 'alpha', 0) throughout the development phase, this results in a lot of 1.2.3.dev0YYYYMMDDHHMMSS-1234abcd with no need for git tags.
• VERSION = (1, 2, 3, 'alpha', 1) for the first alpha release. When it’s tagged and released,

Warning

Do not import anything from the rest of Kolibri in this module, it’s crucial that it can be loaded without the settings/configuration/django stack.

Do not import this file in other package’s __init__, because installation with setup.py should not depend on other packages. In case you were to have a package foo that depended on kolibri, and kolibri is installed as a dependency while foo is installing, then foo won’t be able to access kolibri before after setuptools has completed installation of everything.

Internationalization

As a platform intended for use around the world, Kolibri has a strong mandate for translation and internationalization. As such, it has been designed with technologies to enable this built in.

Writing localized strings

For strings in python files, we are using standard Django tools (gettext and associated functions). See the Django i18n documentation for more information.

For strings in the frontend, we are using Vue-Intl, an in house port of React-intl. Strings are collected during the build process, and bundled into exported JSON files.

Messages will be discovered for any registered plugins and loaded into the page if that language is set as the Django language. All language setting for the front-end are based off the current Django language for the HTTP request.

.vue files

Within Kolibri .vue components, messages are defined in the <script> section as attributes of the component definition:

export default {
  name: 'componentName',
  $trs: {
    msgId1: 'Message text 1',
    msgId2: 'Message text 2',
  },
};

The component names and message IDs should all be camelCase.

User visible strings can be used anywhere in the .vue file using $tr('msgId') (in the template) or this.$tr('msgId') (in the script).

An example Vue component would then look like this

<template>
  <div>
    <!-- puts 'Hello world' in paragraph -->
    <p>{{ $tr('helloWorld') }}</p>
  </div>
</template>


<script>

  export default {
    name: 'someComponent',
    mounted() {
      // prints 'Hello world' to console
      console.log((this.$trs('helloWorld'));
    },
    $trs: {
      helloWorld: 'Hello world',
    },
  };

</script>

.js files

In order to translate strings in Javascript source files, the namespace and messages are defined like this:

import { createTranslator } from 'kolibri.utils.i18n';
const name = 'someModule';
const messages = {
  helloWorld: 'Hello world',
};
const translator = createTranslator(name, messages);

Then messages are available from the $tr method on the translator object:

console.log(translator.$tr('helloWorld'));

ICU message syntax

All front-end translations can be parameterized using ICU message syntax. Additional documentation is available on crowdin.

This syntax can be used to do things like inject variables, pluralize words, and localize numbers.

Dynamic values are passed into translation strings as named arguments in an object. For example:

export default {
  name: 'anothetComponent',
  mounted() {
    // outputs 'Henry read 2 stories'
    console.log(this.$tr('msg', {name: 'Henry', count: 2}));
  },
  $trs: {
    msg: '{name} read {count} {count, plural, one {story} other {stories}}',
  },
};

Crowdin workflow

We use the Crowdin platform to enable third parties to translate the strings in our application.

Note that you have to specify branch names for most commands.

Note

These notes are only for the Kolibri application. For translation of user documentation, please see the kolibri-docs repository.

Prerequisites

First, you’ll need to have these dependencies available on your path:

• GNU gettext
• Java

You may be able to install them using your system’s package manager.

Next, download the crowdin jar to the current directory using this command:

$ make translation-crowdin-install

Finally, ensure you have an environment variable CROWDIN_API_KEY set to the Learning Equality organization account API key.

Note

We do not currently support making translations on Windows. It might be possible, but would require inspection of the Makefile and running alternate commands.

Note

If you install gettext on Mac with Homebrew, you may need to add the binary to your path manually

Exporting and uploading

Typically, strings will be uploaded when a new release branch is cut from develop, signifying the beginning of string freeze and the beta releases.

Before translators can begin working on the strings in our application, they need to be uploaded to Crowdin. Translations are maintained in release branches on Crowdin in the Crowdin kolibri project.

This command will extract front- and backend strings from the Kolibri application code:

$ make translation-extract

And this command will upload the strings to Crowdin:

$ make translation-crowdin-upload branch=[release-branch-name]

The branch name will typically look something like: release-v0.8.x

Next, apply translation memory via the Crowdin UI to copy over perfect matches from previous releases for all languages. Use settings like this:

This could take some time.

Now, some percentage of the newly updated strings should show as already having been translated in crowdin because the perfect matches were automatically copied over.

Fetching and building translations

In order to get the newly translated strings into the application, they need to be downloaded from Crowdin and checked in to the Kolibri github repo.

First, make sure to build the project using the Crowdin website. This is under:

Warning

By default Crowdin will download all translations, not just approved ones. It will often download untranslated strings also. You must manually delete files that are not relevant.

You can download them using this command:

$ make translation-crowdin-download branch=[release-branch-name]

This will update local translation files. Find and delete all files for unsupported or partially-translated languages. Then, check in new strings to git and submit them in a PR to the release branch.

Finally, build the backend .mo files for Django:

$ make translation-django-compilemessages

Adding a newly supported language

In order to add a new supported language to Kolibri, the appropriate language information object must be added to the array in kolibri/locale/supported_languages.json.

The language must be described using the following keys, with everything in lower case

{
  "language_code": "<Two or three letter language code>",
  "language_name": "<Language name in the target language>",
  "territory_code": "<Optional: Language territory code>",
  "script_code": "<Optional: Language script code>"
  "english_name": "<Optional: Language name in English>"
}

For the language names, consult:

1. Primarily, ISO 639 codes
2. Backup reference

Any time a language is added to supported languages the command yarn run generate-locale-data must be run, and the resulting file changes committed to the code base.

If a language is added that is not part of the natively supported languages for Django, then a Django mo file must be compiled for that language using make translation-django-compilemessages, and the resulting mo file committed to the code base.

Other complexities

You may also need to update a few other files to get things to work. In particular:

• the language and territory codes need to match a code defined in the Intl polyfill.
• you may need to update the crowdin.yaml config file to map Crowdin’s language codes to Intl’s language codes
• In some cases, the language doesn’t exist in Django. In these situations, the language needs to be added to EXTRA_LANG_INFO in base.py.

Updating the Perseus plugin

The perseus exercise plugin has its own translated files that also need to be updated when a new language is added:

• Manually download the JSON files for each langage from Crowdin, and add them to the Perseus locale directories with language codes that match Kolibri’s locale folder
• Increment the Perseus version number and publish the new version to PyPi
• Increment the requirements/base.txt file with Perseus’s new version number

Front-end code conventions

Vue.js Components

Note that the top-level tags of Vue.js components are <template>, <script>, and <style>.

• Whitespace
  • an indent is 2 spaces
  • two blank lines between top-level tags
  • one blank line of padding within a top-level tag
  • one level of indent for the contents of all top-level tags
• Keep most child-components stateless. In practice, this means using props but not data.
• Avoid using Vue.js’ camelCase-to-kebab-case mapping. Instead, use square brackets and strings to reference names.
• Use scoped styles where ever possible
• Name custom tags using kebab-case
• Components are placed in the vue directory. The root component file is called vue/index.vue, and is mounted on a tag called <rootvue>.
• Components are defined either as a file with a .vue extension (my-component.vue) or as a directory with an index.vue file (my-component/index.vue). Both forms can be used with require('my-component').
• Put child components inside the directory of a parent component if they are only used by the parent. Otherwise, put shared child components in the vue director.
• Any user visisble interface text should be rendered translatable, see Internationalization for details.

Stylus and CSS

• clear out unused styles
• avoid using classes as JS identifiers, and prefix with js- if necessary

HTML

attribute lists, semantic structure, accessibility…

Files and directories

.cache/…

Testing-related, and ignored by git. TODO - what does it contain?

.eggs/…

Packaging-related, and ignored by git. TODO - what does it contain?

.github/…

These are files used by GitHub to generate templates for things like new pull requests and issues.

.tox/…

Tox is a tool for testing software in a range of environments - for example using different versions of Python and Node.

This directory is ignored by git.

TODO - what does it contain?

dist-packages-cache

Packaging-related, and ignored by git. TODO - what does it contain?

dist-packages-temp

Packaging-related, and ignored by git. TODO - what does it contain?

docs/…

reStructuredText-based documentation, along with Sphinx-based build code

frontend_build/…

Code for integrating Kolibri’s plugin system with webpack instrumentation for bundling client-side dependencies.

karma_config/…

Configuration for Karma, our client-side unit test framework

kolibri/…

main code-base, a Django application

requirements/…

Python dependency files for PIP

test/…

helper files for running tests in Travic CI TODO - is this correct?

.editorconfig

general editor configuration file

.eslintrc.js

configuration file for ESLint, our client-side javascript linter

.gitignore

standard .gitignore file

.htmlhintrc

configuration for our HTML linter, HTMLHint

.pre-commit-config.yaml

configuration for our pre-commit hooks

.stylintrc

configuration for our Stylus linter, Stylint

.travis.yml

configuration for Travis

AUTHORS.rst, CHANGELOG.rst, CONTRIBUTING.rst

reStructuredText-formatted files. Also imported by the generated /docs

LICENSE

plain-text license files

Makefile

wrapper for some scripts, including building packages and docs

MANIFEST.in

list of non-python files to include in the Python package

package.json

javascript dependencies, helper scripts, and configuration

pytest.ini

configuration file for pytest

pytest_runner-2.7.1-py2.7.egg

?

README.rst

reStructuredText-formatted file readme

requirements.txt

Python PIP dependency requirements, simply redirects to requirements/base.txt

setup.cfg

?

setup.py

configuration for Python package related to setuptools

tox.ini

configuration for our Tox test environments

API reference

• Module Index
• Index

Architecture

Front-end architecture

Components

We leverage Vue.js components as the primary building blocks for our UI. For general UI development work, this is the most common tool a developer will use. It would be prudent to read through the Vue.js guide thoroughly.

Each component contains HTML with dynamic Vue.js directives, styling which is scoped to that component (written using Stylus), and logic which is also scoped to that component (all code, including that in Vue components should be written using Bublé compatible ES2015 JavaScript). Non-scoped styles can also be added, but these should be carefully namespaced.

Components allow us to define new custom tags that encapsulate a piece of self-contained, re-usable UI functionality. When composed together, they form a tree structure of parents and children. Each component has a well-defined interface used by its parent component, made up of input properties, events and content slots. Components should never reference their parent.

Read through Front-end code conventions for some important consistency tips on writing new components.

Layout of Frontend Code

Front-end code and assets are generally contained in one of two places: either in one of the plugin subdirectories (under kolibri/plugins) or in kolibri/core, which contains code shared across all plugins as described below.

Within these directories, there should be an assets directory with src and test under it. Most assets will go in src, and tests for the components will go in test.

For example:

kolibri/
  core/                       # core (shared) items
    assets/
      src/
        core-base.vue         # global base template, used by apps
        core-modal.vue        # example of another shared component
        core-global.styl      # globally defined styles, indluded in head
        core-theme.styl       # style variable values
        font-NotoSans.css     # embedded font
      test/
        ...                   # tests for core assets
  plugins/
    learn                     # learn plugin
      assets/
        src/
          vue/
            index.vue         # root view
            some-page.vue     # top-level client-side page
            another-page/     # top-level client-side page
              index.vue
              child.vue       # child component used only by parent
            shared.vue        # shared across this plugin
          app.js              # instantiate learn app on client-side
          router.js
          store.js
        test/
          app.js
    management/
      assets/
        src/
          vue/user-page.vue   # nested-view
          vue/index.vue       # root view
          app.js              # instantiate mgmt app on client-side
        test/
          app.js

In the example above, the vue/another-page/index.vue file in learn can use other assets in the same directory (such as child.vue), components in vue (such as shared.vue), and assets in core (such as variables in core-theme.styl). However it cannot use files in other plugin directories (such as management).

Note

For many development scenarios, only files in these directories need to be touched.

There is also a lot of logic and configuration relevant to front-end code loading, parsing, testing, and linting. This includes webpack, NPM, and integration with the plugin system. This is somewhat scattered, and includes logic in frontend_build/…, package.json, kolibri/core/webpack/…, and other locations. Much of this functionality is described in other sections of the docs (such as Front-end build pipeline), but it can take some time to understand how it all hangs together.

SVG Icons

SVGs can be inlined into Vue components using a special syntax:

<svg src="icon.svg"></svg>

Then, if there is a file called icon.svg in the same directory, that file will be inserted directly into the outputted HTML. This allows aspects of the icon (e.g. fill) to be styled using CSS.

Attributes (such as vue directives like v-if and SVG attributes like viewbox) can also be added to the svg tag.

Single-page Apps

The Kolibri front-end is made of a few high-level “app” plugins, which are single-page JS applications (conventionally app.js) with their own base URL and a single root Vue.js component. Examples of apps are ‘Learn’ and ‘User Management’, as shown in the example above. Apps are independent of each other, and can only reference components and styles from within themselves and from core.

Each app is implemented as a Kolibri plugin and is defined in a subdirectory of kolibri/plugins.

On the Server-side, the kolibri_plugin.py file describes most of the configuration for the single-page app. In particular, this includes the base Django HTML template to return (with an empty <body>), the URL at which the app is exposed, and the javascript entry file which is run on load.

On the client-side, the app creates a single KolibriModule object in the entry file (conventionally app.js) and registers this with the core app, a global variable called kolibriGlobal. The Kolibri Module then mounts single root component to the HTML returned by the server, which recursively contains all additional components, html and logic.

Defining a New Kolibri Module

Note

This section is mostly relevant if you are creating a new app or plugin. If you are just creating new components, you don’t need to do this.

A Kolibri Module is initially defined in Python by sub-classing the WebpackBundleHook class (in kolibri.core.webpack.hooks). The hook defines the JS entry point (conventionally called app.js) where the KolibriModule subclass is instantiated, and where events and callbacks on the module are registered. These are defined in the events and once properties. Each defines key-value pairs of the name of an event, and the name of the method on the KolibriModule object. When these events are triggered on the Kolibri core JavaScript app, these callbacks will be called. (If the KolibriModule is registered for asynchronous loading, the Kolibri Module will first be loaded, and then the callbacks called when it is ready. See Front-end build pipeline for more information.)

All apps should extend the KolibriModule class found in kolibri/core/assets/src/kolibri_module.js.

The ready method will be automatically executed once the Module is loaded and registered with the Kolibri Core App. By convention, JavaScript is injected into the served HTML after the <rootvue> tag, meaning that this tag should be available when the ready method is called, and the root component (conventionally in vue/index.vue) can be mounted here.

Content Renderers

A special kind of Kolibri Module is dedicated to rendering particular content types. All content renderers should extend the ContentRendererModule class found in kolibri/core/assets/src/content_renderer_module.js. In addition, rather than subclassing the WebpackBundleHook class, content renderers should be defined in the Python code using the ContentRendererHook class defined in kolibri.content.hooks. In addition to the standard options for the WebpackBundleHook, the ContentRendererHook also accepts a json file defining the content types that it renders.

Kolibri Content hooks

Hooks for managing the display and rendering of content.

class kolibri.content.hooks.ContentRendererHook(*args, **kwargs)

An inheritable hook that allows special behaviour for a frontend module that defines a content renderer. Reads a JSON file detailing the kinds and file extensions that the renderer can handle.

render_to_page_load_async_html()

Generates script tag containing Javascript to register a content renderer.

Returns:HTML of a script tag to insert into a page.

The ContentRendererModule class has one required property getRendererComponent which should return a Vue component that wraps the content rendering code. This component will be passed defaultFile, files, supplementaryFiles, and thumbnailFiles props, defining the files associated with the piece of content. These can be automatically mixed into a content renderer component definition using the content renderer mixin.

import contentRendererMixin from 'kolibri.coreVue.mixins.contentRenderer';

{
  mixins: [contentRendererMixin],
};

In order to log data about users viewing content, the component should emit startTracking, updateProgress, and stopTracking events, using the Vue $emit method. startTracking and stopTracking are emitted without any arguments, whereas updateProgress should be emitted with a single value between 0 and 1 representing the current proportion of progress on the content.

this.$emit('startTracking');
this.$emit('stopTracking');
this.$emit('updateProgress', 0.25);

For content that has assessment functionality three additional props will be passed: itemId, answerState, and showCorrectAnswer. itemId is a unique identifier for that content for a particular question in the assessment, answerState is passed to prefill an answer (one that has been previously given on an exam, or for a coach to preview a learner’s given answers), showCorrectAnswer is a Boolean that determines if the correct answer for the question should be automatically prefilled without user input - this will only be activated in the case that answerState is falsy - if the renderer is asked to fill in the correct answer, but is unable to do so, it should emit an answerUnavailable event.

The answer renderer should also define a checkAnswer method in its component methods, this method should return an object with the following keys: correct, answerState, and simpleAnswer - describing the correctness, an object describing the answer that can be used to reconstruct it within the renderer, and a simple, human readable answer. If no valid answer is given, null should be returned. In addition to the base content renderer events, assessment items can also emit a hintTaken event to indicate that the user has taken a hint in the assessment, an itemError event to indicate that there has been an error in rendering the requested question corresponding to the itemId, and an interaction event that indicates a user has interacted with the assessment.

import contentRendererMixin from 'kolibri.coreVue.mixins.contentRenderer';

{
  mixins: [contentRendererMixin],
  methods: {
    checkAnswer() {
      return {
        correct: true,
        answerState: {
          answer: 81,
          working: '3^2 = 3 * 3',
        },
        simpleAnswer: '81',
      };
    },
  },
};

Shared Core Functionality

Kolibri provides a set of shared “core” functionality – including components, styles, and helper logic, and libraries – which can be re-used across apps and plugins.

JS Libraries

The following libraries are available globally, in all module code:

• vue - the Vue.js object
• vuex - the Vuex object
• logging - our wrapper around the loglevel logging module
• core-base - a shared base Vue.js component (core-base.vue)

And many others. The complete specification for commonly shared modules can be found in kolibri/core/assets/src/core-app/apiSpec.js - this object defines which modules are imported into the core object. If the module in question has the ‘requireName’ attribute set on the core specification, then it can be used in code with a standard CommonJS-style require statement - e.g.:

const vue = require('kolibri.lib.vue');
const coreBase = require('kolibri.coreVue.components.coreBase');

Adding additional globally-available objects is relatively straightforward due to the plugin and webpack build system.

To expose something on the core app, add a key to the object in apiSpec.js which maps to an object with the following keys:

modulePath: {
    module: require('module-name'),
  }

This module would now be available for import anywhere with the following statement:

const MODULE = require('kolibri.modulePath');

For better organisation of the Core API specification, modules can also be attached at arbitrarily nested paths:

modulePath: {
    nestedPath: {
      module: require('module-name'),
    }
  }

This module would now be available for import anywhere with the following statement:

const MODULE = require('kolibri.modulePath.nestedPath');

For convenience (and to prevent accidental imports), 3rd party (NPM) modules installed in node_modules can be required by their usual name also:

const vue = require('vue');

Bootstrapped Data

The kolibriGlobal object is also used to bootstrap data into the JS app, rather than making unnecessary API requests.

For example, we currently embellish the kolibriGlobal object with a urls object. This is defined by Django JS Reverse and exposes Django URLs on the client side. This will primarily be used for accessing API Urls for synchronizing with the REST API. See the Django JS Reverse documentation for details on invoking the Url.

Styling

For shared styles, two mechanisms are provided:

• The core-theme.styl file provides values for some globally-relevant Stylus variables. These variables can be used in any component’s <style> block by adding the line @require '~core-theme.styl'.
• The core-global.styl file is always inserted into the <head> after normalize.css and provides some basic styling to global elements

Additional Functionality

These methods are also publicly exposed methods of the core app:

kolibriGlobal.register_kolibri_module_async   // Register a Kolibri module for asynchronous loading.
kolibriGlobal.register_kolibri_module_sync    // Register a Kolibri module once it has loaded.
kolibriGlobal.stopListening                   // Unbind an event/callback pair from triggering.
kolibriGlobal.emit                            // Emit an event, with optional args.

Unit Testing

Unit testing is carried out using Mocha. All JavaScript code should have unit tests for all object methods and functions.

Tests are written in JavaScript, and placed in the ‘assets/test’ folder. An example test is shown below:

var assert = require('assert');

var SearchModel = require('../src/search/search_model.js');

describe('SearchModel', function() {
  describe('default result', function() {
    it('should be empty an empty array', function () {
      var test_model = new SearchModel();
      assert.deepEqual(test_model.get("result"), []);
    });
  });
});

Vue.js components can also be tested. The management plugin contains an example (kolibri/plugins/management/assets/test/management.js) where the component is bound to a temporary DOM node, changes are made to the state, and assertions are made about the new component structure.

Adding Dependencies

Dependencies are tracked using yarn - see the docs here.

We distinguish development dependencies from runtime dependencies, and these should be installed as such using yarn add --dev [dep] or yarn add [dep], respectively. Your new dependency should now be recorded in package.json, and all of its dependencies should be recorded in yarn.lock.

Individual plugins can also have their own package.json and yarn.lock for their own dependencies. Running yarn install will also install all the dependencies for each activated plugin (inside a node_modules folder inside the plugin itself). These dependencies will only be available to that plugin at build time. Dependencies for individual plugins should be added from within the root directory of that particular plugin.

To assist in tracking the source of bloat in our codebase, the command yarn run bundle-stats is available to give a full readout of the size that uglified packages take up in the final Javascript code.

In addition, a plugin can have its own webpack.config.js for plugin specific webpack configuration (loaders, plugins, etc.). These options will be merged with the base options using webpack-merge.

Server-client communication

Server API

The Kolibri server represents data as Django Models. These models are defined in models.py files, which can be found in the folders of the different Django apps/plugins.

In Django, Model data are usually exposed to users through webpages that are generated by the Django server. To make the data available to the Kolibri client, which is a single-page app, the Models are exposed as JSON data through a REST API provided by the Django REST Framework (DRF).

In the api.py files, Django REST framework ViewSets are defined which describe how the data is made available through the REST API. Each ViewSet also requires a defined Serializer, which describes the way in which the data from the Django model is serialized into JSON and returned through the REST API. Additionally, optional filters can be applied to the ViewSet which will allow queries to filter by particular features of the data (for example by a field) or by more complex constraints, such as which group the user associated with the data belongs to. Permissions can be applied to a ViewSet, allowing the API to implicitly restrict the data that is returned, based on the currently logged in user.

Finally, in the api_urls.py file, the ViewSets are given a name (through the base_name keyword argument), which sets a particular URL namespace, which is then registered and exposed when the Django server runs. Sometimes, a more complex URL scheme is used, as in the content core app, where every query is required to be prefixed by a channel id (hence the <channel_id> placeholder in that route’s regex pattern)

api_urls.py

router = routers.SimpleRouter()
router.register('content', ChannelMetadataCacheViewSet, base_name="channel")

content_router = routers.SimpleRouter()
content_router.register(r'contentnode', ContentNodeViewset, base_name='contentnode')
content_router.register(r'file', FileViewset, base_name='file')

urlpatterns = [
    url(r'^', include(router.urls)),
    url(r'^content/(?P<channel_id>[^/.]+)/', include(content_router.urls)),
]

To explore the server REST APIs, visit /api_explorer/ on the Kolibri server while running with developer settings.

Client Resource Layer

To access this REST API in the frontend Javascript code, an abstraction layer has been written to reduce the complexity of inferring URLs, caching resources, and saving data back to the server.

Resources

In order to access a particular REST API endpoint, a Javascript Resource has to be defined, an example is shown here

channel.js

const Resource = require('kolibri.lib.apiResource').Resource;

class ChannelResource extends Resource {
  static resourceName() {
    return 'channel';
  }
}

module.exports = ChannelResource;

Here, the resourceName static method must return 'channel' in order to match the base_name assigned to the /content endpoint in api_urls.py.

However, in the case of a more complex endpoint, where arguments are required to form the URL itself (such as in the contentnode endpoints above) - we can add additional required arguments with the resourceIdentifiers static method return value

contentNode.js

const Resource = require('kolibri.lib.apiResource').Resource;

class ContentNodeResource extends Resource {
  static resourceName() {
    return 'contentnode';
  }
  static idKey() {
    return 'pk';
  }
  static resourceIdentifiers() {
    // because ContentNode resources are accessed via
    // /api/content/<channel_id>/contentnode/<pk>
    return [
      'channel_id',
    ];
  }
}

module.exports = ContentNodeResource;

If this resource is part of the core app, it can be added to a global registry of resources inside kolibri/core/assets/src/api-resources/index.js. Otherwise, it can be instantiated as needed, such as in the coach reports module

const ContentSummaryResourceConstructor = require('./apiResources/contentSummary');
const ContentSummaryResource = new ContentSummaryResourceConstructor(coreApp);

First the constructor is imported from the require file, and then an instance is created - with a reference to the Kolibri core app module passed as the only argument.

Models

The instantiated Resource can then be queried for client side representations of particular information. For a representation of a single server side Django model, we can request a Model from the Resource, using getModel

// corresponds to resource address /api/content/<channelId>/contentnode/<id>
const contentModel = ContentNodeResource.getModel(id, { channel_id: channelId });

The first argument is the database id (primary key) for the model, while the second argument defines any additional required resourceIdentifiers that we need to build up the URL.

We now have a reference for a representation of the data on the server. To ensure that it has data from the server, we can call .fetch on it which will resolve to an object representing the data

contentModel.fetch().then((data) => {
  logging.info('This is the model data: ', data);
});

The fetch method returns a Promise which resolves when the data has been successfully retrieved. This may have been due to a round trip call to the REST API, or, if the data has already been previously returned, then it will skip the call to the REST API and return a cached copy of the data.

If you want to pass additional GET parameters to the REST API (to only return a limited set of fields, for example), then you can pass GET parameters in the first argument

contentModel.fetch({ title: true }).then((data) => {
  logging.info('This is the model data: ', data);
});

If it is important to get data that has not been cached, you can call the fetch method with a force parameter

contentModel.fetch({}, true).then((data) => {
  logging.info('This is definitely the most up to date model data: ', data);
});

Collections

For particular views on a data table (which could range from ‘show me everything’ to ‘show me all content nodes with titles starting with “p”’) - Collections are used. Collections are a cached view onto the data table, which are populated by Models - so if a Model that has previously been fetched from the server by a Collection is requested from getModel, it is already cachced.

// corresponds to /api/content/<channelId>/contentnode/?popular=1
const contentCollection = ContentNodeResource.getCollection({ channel_id: channelId }, { popular: 1 });

The first argument defines any additional required resourceIdentifiers that we need to build up the URL, while the second argument defines the GET parameters that are used to define the filters to be applied to the data and hence the subset of the data that the Collection represents.

We now have a reference for a representation of this data on the server. To ensure that it has data from the server, we can call fetch on it, this will resolve to an array of the returned data objects

contentCollection.fetch().then((dataArray) => {
  logging.info('This is the model data: ', dataArray);
});

The fetch method returns a Promise which resolves when the data has been successfully retrieved. This may have been due to a round trip call to the REST API, or, if the data has already been previously returned, then it will skip the call to the REST API and return a cached copy of the data.

If you want to pass additional GET parameters to the REST API (to only return a limited set of fields, for example), then you can pass GET parameters in the first argument

// GET /api/content/<channelId>/contentnode/?popular=1&title=true
contentCollection.fetch({ title: true }).then((dataArray) => {
  logging.info('This is the model data: ', dataArray);
});

If it is important to get data that has not been cached, you can call the fetch method with a force parameter

contentCollection.fetch({}, true).then((dataArray) => {
  logging.info('This is the model data: ', dataArray);
});

Data Flow Diagram

Content database module

This is a core module found in kolibri/Content.

Concepts and Definitions

ContentNode

High level abstraction for prepresenting different content kinds, such as Topic, Video, Audio, Exercise, Document, and can be easily extended to support new content kinds. With multiple ContentNode objects, it supports grouping, arranging them in tree structure, and symmetric and asymmetric relationship between two ContentNode objects.

File

A Django model that is used to store details about the source file, such as what language it supports, how big is the size, which format the file is and where to find the source file.

ContentDB Diagram

• PK = Primary Key
• FK = Foreign Key
• M2M = ManyToManyField

ContentTag

This model is used to establish a filtering system for all ContentNode objects.

ChannelMetadata

A Django model in each content database that stores the database readable names, description and author for each channel.

ChannelMetadataCache

This class stores the channel metadata cached/denormed into the default database.

Implementation Details and Workflows

To achieve using separate databases for each channel and being able to switch channels dynamically, the following data structure and utility functions have been implemented.

ContentDBRoutingMiddleware

This middleware will be applied to every request, and will dynamically select a database based on the channel_id. If a channel ID was included in the URL, it will ensure the appropriate content DB is used for the duration of the request. (Note: set_active_content_database is thread-local, so this shouldn’t interfere with other parallel requests.)

For example, this is how the client side dynamically requests data from a specific channel:

>>> localhost:8000/api/content/<channel_1_id>/contentnode

this will respond with all the contentnode data stored in database <channel_1_id>.sqlite3

>>> localhost:8000/api/content/<channel_2_id>/contentnode

this will respond with all the contentnode data stored in database <channel_2_id>.sqlite3

get_active_content_database

A utility function to retrieve the temporary thread-local variable that using_content_database sets

set_active_content_database

A utility function to set the temporary thread-local variable

using_content_database

A decorator and context manager to do queries on a specific content DB.

Usage as a context manager:

from models import ContentNode

with using_content_database("nalanda"):
    objects = ContentNode.objects.all()
    return objects.count()

Usage as a decorator:

from models import ContentNode

@using_content_database('nalanda')
def delete_all_the_nalanda_content():
    ContentNode.objects.all().delete()

ContentDBRouter

A router that decides what content database to read from based on a thread-local variable.

ContentNode

ContentNode is implemented as a Django model that inherits from two abstract classes, MPTTModel and ContentDatabaseModel. django-mptt’s MPTTModel, which allows for efficient traversal and querying of the ContentNode tree. ContentDatabaseModel is used as a marker so that the content_db_router knows to query against the content database only if the model inherits from ContentDatabaseModel.

The tree structure is established by the parent field that is a foreign key pointing to another ContentNode object. You can also create a symmetric relationship using the related field, or an asymmetric field using the is_prerequisite field.

File

The File model also inherits from ContentDatabaseModel.

To find where the source file is located, the class method get_url uses the checksum field and settings.CONTENT_STORAGE_DIR to calculate the file path. Every source file is named based on its MD5 hash value (this value is also stored in the checksum field) and stored in a namespaced folder under the directory specified in settings.CONTENT_STORAGE_DIR. Because it’s likely to have thousands of content files, and some filesystems cannot handle a flat folder with a large number of files very well, we create namespaced subfolders to improve the performance. So the eventual file path would look something like:

/home/user/.kolibri/content/storage/9/8/9808fa7c560b9801acccf0f6cf74c3ea.mp4

As you can see, it is fine to store your content files outside of the kolibri project folder as long as you set the settings.CONTENT_STORAGE_DIR accordingly.

The front-end will then use the extension field to decide which content player should be used. When the supplementary field’s value is True, that means this File object isn’t necessary and can display the content without it. For example, we will mark caption (subtitle) file as supplementary.

Content Constants

A Python module that stores constants for the kind field in ContentNode model and the preset field and extension field in File model.

Workflows

There are two workflows we currently designed to handle content UI rendering and content playback rendering

• Content UI Rendering

1. Start with a ContentNode object.
2. Get the associated File object that has the thumbnail field being True.
3. Get the thumbnail image by calling this File’s get_url method.
4. Determine the template using the kind field of this ContentNode object.
5. Renders the template with the thumbnail image.

• Content Playback Rendering

1. Start with a ContentNode object.
2. Retrieve a queryset of associated File objects that are filtered by the preset.
3. Use the thumbnail field as a filter on this queryset to get the File object and call this File object’s get_url method to get the source file (the thumbnail image)
4. Use the supplementary field as a filter on this queryset to get the “supplementary” File objects, such as caption (subtitle), and call these File objects’ get_url method to get the source files.
5. Use the supplementary field as a filter on this queryset to get the essential File object. Call its get_url method to get the source file and use its extension field to choose the content player.
6. Play the content.

API Methods

class kolibri.content.api.ChannelMetadataViewSet(**kwargs)

dispatch(request, *args, **kwargs)

.dispatch() is pretty much the same as Django’s regular dispatch, but with extra hooks for startup, finalize, and exception handling.

get_queryset()

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ChannelMetadataSerializer

class kolibri.content.api.ContentNodeFileSizeViewSet(**kwargs)

get_queryset()

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ContentNodeGranularSerializer

class kolibri.content.api.ContentNodeGranularViewset(**kwargs)

get_queryset()

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ContentNodeGranularSerializer

class kolibri.content.api.ContentNodeProgressViewset(**kwargs)

get_queryset()

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

serializer_class

alias of kolibri.content.serializers.ContentNodeProgressSerializer

class kolibri.content.api.ContentNodeSlimViewset(**kwargs)

dispatch(request, *args, **kwargs)

.dispatch() is pretty much the same as Django’s regular dispatch, but with extra hooks for startup, finalize, and exception handling.

get_object(prefetch=True)

Returns the object the view is displaying. You may want to override this if you need to provide non-standard queryset lookups. Eg if objects are referenced using multiple keyword arguments in the url conf.

get_queryset(prefetch=True)

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

pagination_class

alias of OptionalPageNumberPagination

serializer_class

alias of kolibri.content.serializers.ContentNodeSlimSerializer

class kolibri.content.api.ContentNodeViewset(**kwargs)

copies(request, pk=None)

Returns each nodes that has this content id, along with their ancestors.

copies_count(request, **kwargs)

Returns the number of node copies for each content id.

dispatch(request, *args, **kwargs)

.dispatch() is pretty much the same as Django’s regular dispatch, but with extra hooks for startup, finalize, and exception handling.

get_object(prefetch=True)

Returns the object the view is displaying. You may want to override this if you need to provide non-standard queryset lookups. Eg if objects are referenced using multiple keyword arguments in the url conf.

get_queryset(prefetch=True)

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

pagination_class

alias of OptionalPageNumberPagination

serializer_class

alias of kolibri.content.serializers.ContentNodeSerializer

class kolibri.content.api.FileViewset(**kwargs)

get_queryset()

Get the list of items for this view. This must be an iterable, and may be a queryset. Defaults to using self.queryset.

This method should always be used rather than accessing self.queryset directly, as self.queryset gets evaluated only once, and those results are cached for all subsequent requests.

You may want to override this if you need to provide different querysets depending on the incoming request.

(Eg. return a list of items that is specific to the user)

pagination_class

alias of OptionalPageNumberPagination

serializer_class

alias of kolibri.content.serializers.FileSerializer

class kolibri.content.api.OptionalPageNumberPagination

Pagination class that allows for page number-style pagination, when requested. To activate, the page_size argument must be set. For example, to request the first 20 records: ?page_size=20&page=1

class kolibri.content.api.RemoteChannelViewSet(**kwargs)

list(request, *args, **kwargs)

Gets metadata about all public channels on kolibri studio.

retrieve(request, pk=None)

Gets metadata about a channel through a token or channel id.

kolibri.content.api.cache_forever(some_func)

Decorator for patch_response_headers function

API endpoints

request specific content:

>>> localhost:8000/api/content/<channel_id>/contentnode/<content_id>

search content:

>>> localhost:8000/api/content/<channel_id>/contentnode/?search=<search words>

request specific content with specified fields:

>>> localhost:8000/api/content/<channel_id>/contentnode/<content_id>/?fields=pk,title,kind

request paginated contents

>>> localhost:8000/api/content/<channel_id>/contentnode/?page=6&page_size=10

request combines different usages

>>> localhost:8000/api/content/<channel_id>/contentnode/?fields=pk,title,kind,instance_id,description,files&page=6&page_size=10&search=wh

Models

This is one of the Kolibri core components, the abstract layer of all contents. To access it, please use the public APIs in api.py

The ONLY public object is ContentNode

Developer note: If you modify the schema here, it has implications for the content import pipeline, including Kolibri Studio where the imported content databases are created.

Currently, Kolibri Studio has a modified copy of this models.py for generating backwards compatible content databases. Changes here should be propagated to that copy in order to allow for generation of databases with the changed schema. TODO: (rtibbles) achieve this by abstract base models that are instantiated in both applications.

As such, if models or fields are added or removed, or their type is changed, the CONTENT_SCHEMA_VERSION value must be incremented, with an additional constant added for the new version. e.g. a new constant VERSION_3 = ‘3’, should be added, and CONTENT_SCHEMA_VERSION set to VERSION_3.

In addition, the new constant should be added to the mappings dict in ./utils/channel_import.py with an appropriate ChannelImport class associated. The mappings dict is a dict of content schema versions, with an associated ChannelImport class that will allow proper import of that content database into the current content schema for Kolibri.

If the new schema requires inference of the field when it is missing from old databases (i.e. it does not have a default value, or cannot be null or blank), then all the ChannelImport classes for previous versions must be updated to infer this data from old databases.

A pickled SQLAlchemy schema for the new schema must also be generated using the generate_schema management command. This must be generated using an empty, migrated database.

The ‘version’ parameter passed to the command should be the value of e.g. VERSION_3:

kolibri manage generate_schema 3

All pickled schema should be registered in the CONTENT_DB_SCHEMA_VERSIONS list in this file e.g. VERSION_3 should be added to the list.

The channel import test classes for the previous schema should also be added in ./test/test_channel_import.py - it should inherit from the NaiveImportTestCase, and set the name property to the previous CONTENT_SCHEMA_VERSION e.g. VERSION_2.

class kolibri.content.models.AssessmentMetaData(*args, **kwargs)

A model to describe additional metadata that characterizes assessment behaviour in Kolibri. This model contains additional fields that are only revelant to content nodes that probe a user’s state of knowledge and allow them to practice to Mastery. ContentNodes with this metadata may also be able to be used within quizzes and exams.

Parameters:

• id (UUIDField) – Id
• contentnode_id (ForeignKey to ~) – Contentnode
• assessment_item_ids (JSONField) – Assessment item ids
• number_of_assessments (IntegerField) – Number of assessments
• mastery_model (JSONField) – Mastery model
• randomize (BooleanField) – Randomize
• is_manipulable (BooleanField) – Is manipulable

exception DoesNotExist

exception MultipleObjectsReturned

class kolibri.content.models.ChannelMetadata(*args, **kwargs)

Holds metadata about all existing content databases that exist locally.

Parameters:

• id (UUIDField) – Id
• name (CharField) – Name
• description (CharField) – Description
• author (CharField) – Author
• version (IntegerField) – Version
• thumbnail (TextField) – Thumbnail
• last_updated (DateTimeTzField) – Last updated
• min_schema_version (CharField) – Min schema version
• root_id (ForeignKey to ~) – Root

exception DoesNotExist

exception MultipleObjectsReturned

class kolibri.content.models.ContentNode(*args, **kwargs)

The top layer of the contentDB schema, defines the most common properties that are shared across all different contents. Things it can represent are, for example, video, exercise, audio or document…

Parameters:

• id (UUIDField) – Id
• parent_id (TreeForeignKey to ~) – Parent
• license_name (CharField) – License name
• license_description (TextField) – License description
• title (CharField) – Title
• coach_content (BooleanField) – Coach content
• content_id (UUIDField) – Content id
• channel_id (UUIDField) – Channel id
• description (TextField) – Description
• sort_order (FloatField) – Sort order
• license_owner (CharField) – License owner
• author (CharField) – Author
• kind (CharField) – Kind
• available (BooleanField) – Available
• stemmed_metaphone (CharField) – Stemmed metaphone
• lang_id (ForeignKey to ~) – Lang
• lft (PositiveIntegerField) – Lft
• rght (PositiveIntegerField) – Rght
• tree_id (PositiveIntegerField) – Tree id
• level (PositiveIntegerField) – Level
• has_prerequisite (ManyToManyField) – Has prerequisite
• related (ManyToManyField) – Related
• tags (ManyToManyField) – Tags

exception DoesNotExist

exception MultipleObjectsReturned

get_descendant_content_ids()

Retrieve a queryset of content_ids for non-topic content nodes that are descendants of this node.

class kolibri.content.models.ContentTag(id, tag_name)

Parameters:

• id (UUIDField) – Id
• tag_name (CharField) – Tag name

exception DoesNotExist

exception MultipleObjectsReturned

class kolibri.content.models.File(*args, **kwargs)

The second to bottom layer of the contentDB schema, defines the basic building brick for content. Things it can represent are, for example, mp4, avi, mov, html, css, jpeg, pdf, mp3…

Parameters:

• id (UUIDField) – Id
• local_file_id (ForeignKey to ~) – Local file
• available (BooleanField) – Available
• contentnode_id (ForeignKey to ~) – Contentnode
• preset (CharField) – Preset
• lang_id (ForeignKey to ~) – Lang
• supplementary (BooleanField) – Supplementary
• thumbnail (BooleanField) – Thumbnail
• priority (IntegerField) – Priority

exception DoesNotExist

exception MultipleObjectsReturned

get_download_filename()

Return a valid filename to be downloaded as.

get_download_url()

Return the download url.

get_preset()

Return the preset.

class kolibri.content.models.Language(id, lang_code, lang_subcode, lang_name, lang_direction)

Parameters:

• id (CharField) – Id
• lang_code (CharField) – Lang code
• lang_subcode (CharField) – Lang subcode
• lang_name (CharField) – Lang name
• lang_direction (CharField) – Lang direction

exception DoesNotExist

exception MultipleObjectsReturned

class kolibri.content.models.LocalFile(*args, **kwargs)

The bottom layer of the contentDB schema, defines the local state of files on the device storage.

Parameters:

• id (CharField) – Id
• extension (CharField) – Extension
• available (BooleanField) – Available
• file_size (IntegerField) – File size

exception DoesNotExist

exception MultipleObjectsReturned

get_storage_url()

Return a url for the client side to retrieve the content file. The same url will also be exposed by the file serializer.

class kolibri.content.models.UUIDField(*args, **kwargs)

Adaptation of Django’s UUIDField, but with 32-char hex representation as Python representation rather than a UUID instance.

deconstruct()

Returns enough information to recreate the field as a 4-tuple:

• The name of the field on the model, if contribute_to_class has been run
• The import path of the field, including the class: django.db.models.IntegerField This should be the most portable version, so less specific may be better.
• A list of positional arguments
• A dict of keyword arguments

Note that the positional or keyword arguments must contain values of the following types (including inner values of collection types):

• None, bool, str, unicode, int, long, float, complex, set, frozenset, list, tuple, dict
• UUID
• datetime.datetime (naive), datetime.date
• top-level classes, top-level functions - will be referenced by their full import path
• Storage instances - these have their own deconstruct() method

This is because the values here must be serialized into a text format (possibly new Python code, possibly JSON) and these are the only types with encoding handlers defined.

There’s no need to return the exact way the field was instantiated this time, just ensure that the resulting field is the same - prefer keyword arguments over positional ones, and omit parameters with their default values.

get_db_prep_value(value, connection, prepared=False)

Returns field’s value prepared for interacting with the database backend.

Used by the default implementations of get_db_prep_save().

to_python(value)

Converts the input value into the expected Python data type, raising django.core.exceptions.ValidationError if the data can’t be converted. Returns the converted value. Subclasses should override this.

Users, auth, and permissions module

This is a core module found in kolibri/auth.

Concepts and Definitions

Facility

All user data (accounts, logs, ratings, etc) in Kolibri are associated with a particular “Facility”. A Facility is a grouping of users who are physically co-located, and who generally access Kolibri from the same server on a local network, for example in a school, library, or community center. Collectively, all the data associated with a particular Facility are referred to as a “Facility Dataset”.

Users

There are two kinds of users: FacilityUser and DeviceOwner. A FacilityUser is associated with a particular Facility, and the user’s account and data may be synchronized across multiple devices. A DeviceOwner account is not associated with a particular Facility, but is specific to one device, and is never synchronized across multiple devices. A DeviceOwner is like a superuser, and has permissions to modify any data on her own device, whereas a FacilityUser only has permissions for some subset of data from their own Facility Dataset (as determined in part by the roles they possess; see below).

Collections

Collections are hierarchical groups of users, used for grouping users and making decisions about permissions. Users can have roles for one or more Collections, by way of obtaining Roles associated with those Collections. Collections can belong to other Collections, and user membership in a collection is conferred through Membership. Collections are subdivided into several pre-defined levels: Facility, Classroom, and LearnerGroup, as illustrated here:

In this illustration, Facility X contains two Classrooms, Class A and Class B. Class A contains two LearnerGroups, Group Q and Group R.

Membership

A FacilityUser (but not a DeviceOwner) can be marked as a member of a Collection through a Membership object. Being a member of a Collection also means being a member of all the Collections above that Collection in the hierarchy. Thus, in the illustration below, Alice is directly associated with Group Q through a Membership object, which makes her a member of Group Q. As Group Q is contained within Class A, which is contained within Facility X, she is also implicitly a member of both those collections.

Note also that a FacilityUser is always implicitly a member of the Facility with which it is associated, even if it does not have any Membership objects.

Roles

Another way in which a FacilityUser can be associated with a particular Collection is through a Role object, which grants the user a role with respect to the Collection and all the collections below it. A Role object also stores the “kind” of the role (currently, one of “admin” or “coach”), which affects what permissions the user gains through the Role.

To illustrate, consider the example in the following figure:

The figure shows a Role object linking Bob with Class A, and the Role is marked with kind “coach”, which we can informally read as “Bob is a coach for Class A”. We consider user roles to be “downward-transitive” (meaning if you have a role for a collection, you also have that role for descendents of that collection). Thus, in our example, we can say that “Bob is also a coach for Group Q”. Furthermore, as Alice is a member of Group Q, we can say that “Bob is a coach for Alice”.

Role-Based Permissions

As a lot of Facility Data in Kolibri is associated with a particular FacilityUser, for many objects we can concisely define a requesting user’s permissions in terms of his or her roles for the object’s associated User. For example, if a ContentLog represents a particular FacilityUser’s interaction with a piece of content, we might decide that another FacilityUser can view the ContentLog if she is a coach (has the coach role) for the user. In our scenario above, this would mean that Bob would have read permissions for a ContentLog for which “user=Alice”, by virtue of having the coach role for Alice.

Some data may not be related to a particular user, but rather with a Collection (e.g. the Collection object itself, settings for a Collection, or content assignments for a Collection). Permissions for these objects can be defined in terms of the role the requesting User has with respect to the object’s associated Collection. So, for example, we might allow Bob to assign content to Class A on the basis of him having the “coach” role for Class A.

Permission Levels

As we are constructing a RESTful API for accessing data within Kolibri, the core actions for which we need to define permissions are the CRUD operations (Create, Read, Update, Delete). As Create, Update, and Delete permissions often go hand in hand, we can collectively refer to them as “Write Permissions”.

Implementation Details

Collections

A Collection is implemented as a Django model that inherits from django-mptt’s MPTTModel, which allows for efficient traversal and querying of the collection hierarchy. For convenience, the specific types of collections – Facility, Classroom, and LearnerGroup – are implemented as _proxy models of the main Collection model. There is a kind field on Collection that allows us to distinguish between these types, and the ModelManager for the proxy models returns only instances of the matching kind.

From a Collection instance, you can traverse upwards in the tree with the parent field, and downwards via the children field (which is a reverse RelatedManager for the parent field):

>>> my_classroom.parent
<Collection: "Facility X" (facility)>

>>> my_facility.children.all()
[<Collection: "Class A" (classroom)>, <Collection: "Class B" (classroom)>]

Note that the above methods (which are provided by MPTTModel) return generic Collection instances, rather than specific proxy model instances. To retrieve parents and children as appropriate proxy models, use the helper methods provided on the proxy models, e.g.:

>>> my_classroom.get_facility()
<Facility: Facility X>

>>> my_facility.get_classrooms()
[<Classroom: Class A>, <Classroom: Class B>]

Facility and FacilityDataset

The Facility model (a proxy model for Collection, as described above) is special in that it has no parent; it is the root of a tree. A Facility model instance, and all other Facility Data associated with the Facility and its FacilityUsers, inherits from AbstractFacilityDataModel, which has a dataset field that foreign keys onto a common FacilityDataset instance. This makes it easy to check, for purposes of permissions or filtering data for synchronization, which instances are part of a particular Facility Dataset. The dataset field is automatically set during the save method, by calling the infer_dataset method, which must be overridden in every subclass of AbstractFacilityDataModel to return the dataset to associate with that instance.

Efficient Hierarchy Calculations

In order to make decisions about whether a user has a certain permission for an object, we need an efficient way to retrieve the set of roles the user has in relation to that object. This involves traversing the Role table, Collection hierarchy, and possibly the Membership table, but we can delegate most of the work to the database engine (and leverage efficient hierarchy lookups afforded by MPTT). The following algorithms and explanations will refer to the naming in the following diagram:

In pseudocode, the query for “What Roles does Source User have in relation to Target User?” would be implemented in the following way:

Fetch all Roles with:
    User: Source User
    Collection: Ancestor Collection
For which there is a Membership with:
    User: Target User
    Collection: Descendant Collection
And where:
    Ancestor Collection is an ancestor of (or equal to) Descendant Collection

At the database level, this can be written in the following way, as a single multi-table SQL query:

SELECT DISTINCT
    source_role.kind
FROM
    collection_table AS ancestor_coll,
    collection_table AS descendant_coll,
    role_table,
    membership_table
WHERE
    role_table.user_id = {source_user_id} AND
    role_table.collection_id = ancestor_coll.id AND
    membership_table.user_id = {target_user_id}
    membership_table.collection_id = descendant_coll.id AND
    descendant_coll.lft BETWEEN ancestor_coll.lft AND ancestor_coll.rght AND
    descendant_coll.tree_id = ancestor_coll.tree_id;

Similarly, performing a queryset filter like “give me all ContentLogs associated with FacilityUsers for which Source User has an admin role” can be written as:

SELECT
    contentlog_table.*
FROM
    contentlog_table
WHERE EXISTS
    (SELECT
         *
     FROM
         collection_table AS ancestor_coll,
         collection_table AS descendant_coll,
         role_table,
         membership_table
     WHERE
         role_table.user_id = {source_user_id} AND
         role_table.collection_id = ancestor_coll.id AND
         membership_table.user_id = contentlog_table.user_id
         membership_table.collection_id = descendant_coll.id AND
         descendant_coll.lft BETWEEN ancestor_coll.lft AND ancestor_coll.rght AND
         descendant_coll.tree_id = ancestor_coll.tree_id
    )

Note the membership_table.user_id = contentlog_table.user_id condition, which links the role-membership-collection hierarchy subquery into the main query. We refer to this condition as the “anchor”.

To facilitate making queries that leverage the role-membership-collection hierarchy, without needing to write custom SQL each time, we have implemented a HierarchyRelationsFilter helper class. The class is instantiated by passing in a queryset, and then exposes a filter_by_hierarchy method that allows various parts of the role-membership-collection hierarchy to be constrained, and anchored back into the queryset’s main table. It then returns a filtered queryset (with appropriate conditions applied) upon which further filters or other queryset operations can be applied.

The signature for filter_by_hierarchy is:

def filter_by_hierarchy(self,
                        source_user=None,
                        role_kind=None,
                        ancestor_collection=None,
                        descendant_collection=None,
                        target_user=None):

With the exception of role_kind (which is either a string or list of strings, of role kinds), these parameters accept either:

• A model instance (either a FacilityUser or a Collection subclass, as appropriate) or its ID
• An F expression that anchors some part of the hierarchy back into the base queryset model (the simplest usage is just to put the name of a field from the base model in the F function, but you can also indirectly reference fields of related models, e.g. F("collection__parent"))

For example, the ContentLog query described above (“give me all ContentLogs associated with FacilityUsers for which Source User has an admin role”) can be implemented as:

contentlogs = HierarchyRelationsFilter(ContentLog.objects.all()).filter_by_hierarchy(
    source_user=my_source_user,  # specify the specific user to be the source user
    role_kind=role_kinds.ADMIN,  # make sure the Role is an admin role
    target_user=F("user"),  # anchor the target user to the "user" field of the ContentLog model
)

Managing Roles and Memberships

User and Collection models have various helper methods for retrieving and modifying roles and memberships:

• To get all the members of a collection (including those of its descendant collections), use Collection.get_members().
• To add or remove roles/memberships, use the add_role, remove_role, add_member, and remove_member methods of Collection (or the additional convenience methods, such as add_admin, that exist on the proxy models).
• To check whether a user is a member of a Collection, use KolibriAbstractBaseUser.is_member_of (for DeviceOwner, this always returns False)
• To check whether a user has a particular kind of role for a collection or another user, use the has_role_for_collection and has_role_for_user methods of KolibriAbstractBaseUser.
• To list all role kinds a user has for a collection or another user, use the get_roles_for_collection and get_roles_for_user methods of KolibriAbstractBaseUser.

Encoding Permission Rules

We need to associate a particular set of rules with each model, to specify the permissions that users should have in relation to instances of that model. While not all models have the same rules, there are some broad categories of models that do share the same rules (e.g. ContentInteractionLog, ContentSummaryLog, and UserSessionLog – collectively, “User Log Data”). Hence, it is useful to encapsulate a permissions “class” that can be reused across models, and extended (through inheritance) if slightly different behavior is needed. These classes of permissions are defined as Python classes that inherit from kolibri.auth.permissions.base.BasePermissions, which defines the following overridable methods:

• The following four Boolean (True/False) permission checks, corresponding to the “CRUD” operations: - user_can_create_object - user_can_read_object - user_can_update_object - user_can_delete_object
• The queryset-filtering readable_by_user_filter method, which takes in a queryset and returns a queryset filtered down to just objects that should be readable by the user.

Associating Permissions with Models

A model is associated with a particular permissions class through a “permissions” attribute defined on the top level of the model class, referencing an instance of a Permissions class (a class that subclasses BasePermissions). For example, to specify that a model ContentSummaryLog should draw its permissions rules from the UserLogPermissions class, modify the model definition as follows:

class ContentSummaryLog(models.Model):

    permissions = UserLogPermissions()

    <remainder of model definition>

Specifying Role-Based Permissions

Defining a custom Permissions class and overriding its methods allows for arbitrary logic to be used in defining the rules governing the permissions, but many cases can be covered by more constrained rule specifications. In particular, the rules for many models can be specified in terms of the role- based permissions system described above. A built-in subclass of BasePermissions, called RoleBasedPermissions, makes this easy. Creating an instance of RoleBasedPermissions involves passing in the following parameters:

• Tuples of role kinds that should be granted each of the CRUD permissions, encoded in the following parameters: can_be_created_by, can_be_read_by, can_be_updated_by, can_be_deleted_by.
• The target_field parameter that determines the “target” object for the role-checking; this should be the name of a field on the model that foreign keys either onto a FacilityUser or a Collection. If the model we’re checking permissions for is itself the target, then target_field may be ".".

An example, showing that read permissions should be granted to a coach or admin for the user referred to by the model’s “user” field. Similarly, write permissions should only be available to an admin for the user:

class UserLog(models.Model):

    permissions = RoleBasedPermissions(
        target_field="user",
        can_be_created_by=(role_kinds.ADMIN,),
        can_be_read_by=(role_kinds.COACH, role_kinds.ADMIN),
        can_be_updated_by=(role_kinds.ADMIN,),
        can_be_deleted_by=(role_kinds.ADMIN,),
    )

    <remainder of model definition>

Built-in Permissions Classes

Some common rules are encapsulated by the permissions classes in kolibri.auth.permissions.general. These include:

• IsOwn: only allows access to the object if the object belongs to the requesting user (in other words, if the object has a specific field, field_name, that foreign keys onto the user)
• IsFromSameFacility: only allows access to object if user is associated with the same facility as the object
• IsSelf: only allows access to the object if the object is the user

A general pattern with these provided classes is to allow an argument called read_only, which means that rather than allowing both write (create, update, delete) and read permissions, they will only grant read permission. So, for example, IsFromSameFacility(read_only=True) will allow any user from the same facility to read the model, but not to write to it, whereas IsFromSameFacility(read_only=False) or IsFromSameFacility() would allow both.

Combining Permissions Classes

In many cases, it may be necessary to combine multiple permission classes together to define the ruleset that you want. This can be done using the Boolean operators | (OR) and & (AND). So, for example, IsOwn(field_name="user") | IsSelf() would allow access to the model if either the model has a foreign key named “user” that points to the user, or the model is itself the user model. Combining two permission classes with &, on the other hand, means both classes must return True for a permission to be granted. Note that permissions classes combined in this way still support the readable_by_user_filter method, returning a queryset that is either the union (for |) or intersection (&) of the querysets that were returned by each of the permissions classes.

Checking Permissions

Checking whether a user has permission to perform a CRUD operation on an object involves calling the appropriate methods on the KolibriAbstractBaseUser (FacilityUser or DeviceOwner) instance. For instance, to check whether request.user has delete permission for ContentSummaryLog instance log_obj, you could do:

if request.user.can_delete(log_obj):
    log_obj.delete()

Checking whether a user can create an object is slightly different, as you may not yet have an instance of the model. Instead, pass in the model class and a dict of the data that you want to create it with:

data = {"user": request.user, "content_id": "qq123"}
if request.user.can_create(ContentSummaryLog, data):
    ContentSummaryLog.objects.create(**data)

To efficiently filter a queryset so that it only includes records that the user should have permission to read (to make sure you’re not sending them data they shouldn’t be able to access), use the filter_readable method:

all_results = ContentSummaryLog.objects.filter(content_id="qq123")
permitted_results = request.user.filter_readable(all_results)

Note that for the DeviceOwner model, these methods will simply return True (or unfiltered querysets), as device owners are considered superusers. For the FacilityUser model, they defer to the permissions encoded in the permission object on the model class.

Using Kolibri Permissions with Django REST Framework

There are two classes that make it simple to leverage the permissions system described above within a Django REST Framework ViewSet, to restrict permissions appropriately on API endpoints, based on the currently logged-in user.

KolibriAuthPermissions is a subclass of rest_framework.permissions.BasePermission that defers to our KolibriAbstractBaseUser permissions interface methods for determining which object-level permissions to grant to the current user:

• Permissions for ‘POST’ are based on request.user.can_create
• Permissions for ‘GET’, ‘OPTIONS’ and ‘HEAD’ are based on request.user.can_read (Note that adding KolibriAuthPermissions only checks object-level permissions, and does not filter queries made against a list view; see KolibriAuthPermissionsFilter below)
• Permissions for ‘PUT’ and ‘PATCH’ are based on request.user.can_update
• Permissions for ‘DELETE’ are based on request.user.can_delete

KolibriAuthPermissions is a subclass of rest_framework.filters.BaseFilterBackend that filters list views to include only records for which the current user has read permissions. This only applies to ‘GET’ requests.

For example, to use the Kolibri permissions system to restrict permissions for an API endpoint providing access to a ContentLog model, you would do the following:

from kolibri.auth.api import KolibriAuthPermissions, KolibriAuthPermissionsFilter

class FacilityViewSet(viewsets.ModelViewSet):
    permission_classes = (KolibriAuthPermissions,)
    filter_backends = (KolibriAuthPermissionsFilter,)
    queryset = ContentLog.objects.all()
    serializer_class = ContentLogSerializer

Models

We have four main abstractions: Users, Collections, Memberships, and Roles.

Users represent people, like students in a school, teachers for a classroom, or volunteers setting up informal installations. A FacilityUser belongs to a particular facility, and has permissions only with respect to other data that is associated with that facility. FacilityUser accounts (like other facility data) may be synced across multiple devices.

Collections form a hierarchy, with Collections able to belong to other Collections. Collections are subdivided into several pre-defined levels (Facility > Classroom > LearnerGroup).

A FacilityUser (but not a DeviceOwner) can be marked as a member of a Collection through a Membership object. Being a member of a Collection also means being a member of all the Collections above that Collection in the hierarchy.

Another way in which a FacilityUser can be associated with a particular Collection is through a Role object, which grants the user a role with respect to the Collection and all the collections below it. A Role object also stores the “kind” of the role (currently, one of “admin” or “coach”), which affects what permissions the user gains through the Role.

class kolibri.auth.models.AbstractFacilityDataModel(*args, **kwargs)

Base model for Kolibri “Facility Data”, which is data that is specific to a particular Facility, such as FacilityUsers, Collections, and other data associated with those users and collections.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

clean_fields(*args, **kwargs)

Cleans all fields and raises a ValidationError containing a dict of all validation errors if any occur.

ensure_dataset(*args, **kwargs)

If no dataset has yet been specified, try to infer it. If a dataset has already been specified, to prevent inconsistencies, make sure it matches the inferred dataset, otherwise raise a KolibriValidationError. If we have no dataset and it can’t be inferred, we raise a KolibriValidationError exception as well.

full_clean(*args, **kwargs)

Calls clean_fields, clean, and validate_unique, on the model, and raises a ValidationError for any errors that occurred.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

save(*args, **kwargs)

Saves the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

class kolibri.auth.models.Classroom(id, _morango_dirty_bit, _morango_source_id, _morango_partition, dataset, name, parent, kind, lft, rght, tree_id, level)

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• name (CharField) – Name
• parent_id (TreeForeignKey to ~) – Parent
• kind (CharField) – Kind
• lft (PositiveIntegerField) – Lft
• rght (PositiveIntegerField) – Rght
• tree_id (PositiveIntegerField) – Tree id
• level (PositiveIntegerField) – Level

exception DoesNotExist

exception MultipleObjectsReturned

get_facility()

Gets the Classroom’s parent Facility.

Returns:A Facility instance.

get_learner_groups()

Returns a QuerySet of LearnerGroups associated with this Classroom.

Returns:A LearnerGroup QuerySet.

save(*args, **kwargs)

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.Collection(*args, **kwargs)

Collections are hierarchical groups of FacilityUsers, used for grouping users and making decisions about permissions. FacilityUsers can have roles for one or more Collections, by way of obtaining Roles associated with those Collections. Collections can belong to other Collections, and user membership in a Collection is conferred through Memberships. Collections are subdivided into several pre-defined levels.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• name (CharField) – Name
• parent_id (TreeForeignKey to ~) – Parent
• kind (CharField) – Kind
• lft (PositiveIntegerField) – Lft
• rght (PositiveIntegerField) – Rght
• tree_id (PositiveIntegerField) – Tree id
• level (PositiveIntegerField) – Level

exception DoesNotExist

exception MultipleObjectsReturned

add_member(user)

Create a Membership associating the provided user with this Collection. If the Membership object already exists, just return that, without changing anything.

Parameters:user – The FacilityUser to add to this Collection. Returns:The Membership object (possibly new) that associates the user with the Collection.

add_role(user, role_kind)

Create a Role associating the provided user with this collection, with the specified kind of role. If the Role object already exists, just return that, without changing anything.

Parameters:

• user – The FacilityUser to associate with this Collection.
• role_kind – The kind of role to give the user with respect to this Collection.

Returns:

The Role object (possibly new) that associates the user with the Collection.

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

clean_fields(*args, **kwargs)

Cleans all fields and raises a ValidationError containing a dict of all validation errors if any occur.

get_coaches()

Returns users who have the coach role for this immediate collection.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

remove_member(user)

Remove any Membership objects associating the provided user with this Collection.

Parameters:user – The FacilityUser to remove from this Collection. Returns:True if a Membership was removed, False if there was no matching Membership to remove.

remove_role(user, role_kind)

Remove any Role objects associating the provided user with this Collection, with the specified kind of role.

Parameters:

• user – The FacilityUser to dissociate from this Collection (for the specific role kind).
• role_kind – The kind of role to remove from the user with respect to this Collection.

save(*args, **kwargs)

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.CollectionProxyManager

get_queryset()

Ensures that this manager always returns nodes in tree order.

class kolibri.auth.models.Facility(id, _morango_dirty_bit, _morango_source_id, _morango_partition, dataset, name, parent, kind, lft, rght, tree_id, level)

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• name (CharField) – Name
• parent_id (TreeForeignKey to ~) – Parent
• kind (CharField) – Kind
• lft (PositiveIntegerField) – Lft
• rght (PositiveIntegerField) – Rght
• tree_id (PositiveIntegerField) – Tree id
• level (PositiveIntegerField) – Level

exception DoesNotExist

exception MultipleObjectsReturned

ensure_dataset(*args, **kwargs)

If no dataset has yet been specified, try to infer it. If a dataset has already been specified, to prevent inconsistencies, make sure it matches the inferred dataset, otherwise raise a KolibriValidationError. If we have no dataset and it can’t be inferred, we raise a KolibriValidationError exception as well.

get_classrooms()

Returns a QuerySet of Classrooms under this Facility.

Returns:A Classroom QuerySet.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

save(*args, **kwargs)

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.FacilityDataSyncableModel(*args, **kwargs)

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition

class kolibri.auth.models.FacilityDataset(*args, **kwargs)

FacilityDataset stores high-level metadata and settings for a particular Facility. It is also the model that all models storing facility data (data that is associated with a particular facility, and that inherits from AbstractFacilityDataModel) foreign key onto, to indicate that they belong to this particular Facility.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• description (TextField) – Description
• location (CharField) – Location
• preset (CharField) – Preset
• learner_can_edit_username (BooleanField) – Learner can edit username
• learner_can_edit_name (BooleanField) – Learner can edit name
• learner_can_edit_password (BooleanField) – Learner can edit password
• learner_can_sign_up (BooleanField) – Learner can sign up
• learner_can_delete_account (BooleanField) – Learner can delete account
• learner_can_login_with_no_password (BooleanField) – Learner can login with no password
• show_download_button_in_learn (BooleanField) – Show download button in learn

exception DoesNotExist

exception MultipleObjectsReturned

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

class kolibri.auth.models.FacilityUser(*args, **kwargs)

FacilityUser is the fundamental object of the auth app. These users represent the main users, and can be associated with a hierarchy of Collections through Memberships and Roles, which then serve to help determine permissions.

Parameters:

• password (CharField) – Password
• last_login (DateTimeField) – Last login
• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• username (CharField) – Required. 30 characters or fewer. Letters and digits only
• full_name (CharField) – Full name
• date_joined (DateTimeTzField) – Date joined
• facility_id (ForeignKey to ~) – Facility

exception DoesNotExist

exception MultipleObjectsReturned

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

can_create_instance(obj)

Checks whether this user (self) has permission to create a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

In general, unless an instance has already been initialized, this method should not be called directly; instead, it should be preferred to call can_create.

Parameters:obj – An (unsaved) instance of a Django model, to check permissions for. Returns:True if this user should have permission to create the object, otherwise False. Return type:bool

can_delete(obj)

Checks whether this user (self) has permission to delete a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to delete the object, otherwise False. Return type:bool

can_manage_content

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

can_read(obj)

Checks whether this user (self) has permission to read a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to read the object, otherwise False. Return type:bool

can_update(obj)

Checks whether this user (self) has permission to update a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to update the object, otherwise False. Return type:bool

filter_readable(queryset)

Filters a queryset down to only the elements that this user should have permission to read.

Parameters:queryset – A QuerySet instance that the filtering should be applied to. Returns:Filtered QuerySet including only elements that are readable by this user.

get_roles_for_collection(coll)

Determine all the roles this user has in relation to the specified Collection, and return a set containing the kinds of roles.

Parameters:coll – The target Collection for which this user has the roles. Returns:The kinds of roles this user has with respect to the specified Collection. Return type:set of kolibri.auth.constants.role_kinds.* strings

get_roles_for_user(user)

Determine all the roles this user has in relation to the target user, and return a set containing the kinds of roles.

Parameters:user – The target user for which this user has the roles. Returns:The kinds of roles this user has with respect to the target user. Return type:set of kolibri.auth.constants.role_kinds.* strings

has_role_for_collection(kinds, coll)

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified Collection.

Parameters:

• kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
• coll – The target Collection for which this user has the roles.

Returns:

True if this user has the specified role kind with respect to the target Collection, otherwise False.

Return type:

bool

has_role_for_user(kinds, user)

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified user.

Parameters:

• user – The user that is the target of the role (for which this user has the roles).
• kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.

Returns:

True if this user has the specified role kind with respect to the target user, otherwise False.

Return type:

bool

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

is_member_of(coll)

Determine whether this user is a member of the specified Collection.

Parameters:coll – The Collection for which we are checking this user’s membership. Returns:True if this user is a member of the specified Collection, otherwise False. Return type:bool

is_staff

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

is_superuser

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

class kolibri.auth.models.KolibriAbstractBaseUser(*args, **kwargs)

Our custom user type, derived from AbstractBaseUser as described in the Django docs. Draws liberally from django.contrib.auth.AbstractUser, except we exclude some fields we don’t care about, like email.

This model is an abstract model, and is inherited by FacilityUser.

Parameters:

• password (CharField) – Password
• last_login (DateTimeField) – Last login
• username (CharField) – Required. 30 characters or fewer. Letters and digits only
• full_name (CharField) – Full name
• date_joined (DateTimeTzField) – Date joined

can_create(Model, data)

Checks whether this user (self) has permission to create an instance of Model with the specified attributes (data).

This method defers to the can_create_instance method, and in most cases should not itself be overridden.

Parameters:

• Model – A subclass of django.db.models.Model
• data – A dict of data to be used in creating an instance of the Model

Returns:

True if this user should have permission to create an instance of Model with the specified data, else False.

Return type:

bool

can_create_instance(obj)

Checks whether this user (self) has permission to create a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

In general, unless an instance has already been initialized, this method should not be called directly; instead, it should be preferred to call can_create.

Parameters:obj – An (unsaved) instance of a Django model, to check permissions for. Returns:True if this user should have permission to create the object, otherwise False. Return type:bool

can_delete(obj)

Checks whether this user (self) has permission to delete a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to delete the object, otherwise False. Return type:bool

can_read(obj)

Checks whether this user (self) has permission to read a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to read the object, otherwise False. Return type:bool

can_update(obj)

Checks whether this user (self) has permission to update a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to update the object, otherwise False. Return type:bool

filter_readable(queryset)

Filters a queryset down to only the elements that this user should have permission to read.

Parameters:queryset – A QuerySet instance that the filtering should be applied to. Returns:Filtered QuerySet including only elements that are readable by this user.

get_roles_for(obj)

Helper function that defers to get_roles_for_user or get_roles_for_collection based on the type of object passed in.

get_roles_for_collection(coll)

Determine all the roles this user has in relation to the specified Collection, and return a set containing the kinds of roles.

Parameters:coll – The target Collection for which this user has the roles. Returns:The kinds of roles this user has with respect to the specified Collection. Return type:set of kolibri.auth.constants.role_kinds.* strings

get_roles_for_user(user)

Determine all the roles this user has in relation to the target user, and return a set containing the kinds of roles.

Parameters:user – The target user for which this user has the roles. Returns:The kinds of roles this user has with respect to the target user. Return type:set of kolibri.auth.constants.role_kinds.* strings

has_role_for(kinds, obj)

Helper function that defers to has_role_for_user or has_role_for_collection based on the type of object passed in.

has_role_for_collection(kinds, coll)

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified Collection.

Parameters:

• kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
• coll – The target Collection for which this user has the roles.

Returns:

True if this user has the specified role kind with respect to the target Collection, otherwise False.

Return type:

bool

has_role_for_user(kinds, user)

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified user.

Parameters:

• user – The user that is the target of the role (for which this user has the roles).
• kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.

Returns:

True if this user has the specified role kind with respect to the target user, otherwise False.

Return type:

bool

is_member_of(coll)

Determine whether this user is a member of the specified Collection.

Parameters:coll – The Collection for which we are checking this user’s membership. Returns:True if this user is a member of the specified Collection, otherwise False. Return type:bool

class kolibri.auth.models.KolibriAnonymousUser

Custom anonymous user that also exposes the same interface as KolibriAbstractBaseUser, for consistency.

Parameters:

• password (CharField) – Password
• last_login (DateTimeField) – Last login
• full_name (CharField) – Full name
• date_joined (DateTimeTzField) – Date joined

can_create_instance(obj)

Checks whether this user (self) has permission to create a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

In general, unless an instance has already been initialized, this method should not be called directly; instead, it should be preferred to call can_create.

Parameters:obj – An (unsaved) instance of a Django model, to check permissions for. Returns:True if this user should have permission to create the object, otherwise False. Return type:bool

can_delete(obj)

Checks whether this user (self) has permission to delete a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to delete the object, otherwise False. Return type:bool

can_read(obj)

Checks whether this user (self) has permission to read a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to read the object, otherwise False. Return type:bool

can_update(obj)

Checks whether this user (self) has permission to update a particular model instance (obj).

This method should be overridden by classes that inherit from KolibriAbstractBaseUser.

Parameters:obj – An instance of a Django model, to check permissions for. Returns:True if this user should have permission to update the object, otherwise False. Return type:bool

filter_readable(queryset)

Filters a queryset down to only the elements that this user should have permission to read.

Parameters:queryset – A QuerySet instance that the filtering should be applied to. Returns:Filtered QuerySet including only elements that are readable by this user.

get_roles_for_collection(coll)

Determine all the roles this user has in relation to the specified Collection, and return a set containing the kinds of roles.

Parameters:coll – The target Collection for which this user has the roles. Returns:The kinds of roles this user has with respect to the specified Collection. Return type:set of kolibri.auth.constants.role_kinds.* strings

get_roles_for_user(user)

Determine all the roles this user has in relation to the target user, and return a set containing the kinds of roles.

Parameters:user – The target user for which this user has the roles. Returns:The kinds of roles this user has with respect to the target user. Return type:set of kolibri.auth.constants.role_kinds.* strings

has_role_for_collection(kinds, coll)

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified Collection.

Parameters:

• kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.
• coll – The target Collection for which this user has the roles.

Returns:

True if this user has the specified role kind with respect to the target Collection, otherwise False.

Return type:

bool

has_role_for_user(kinds, user)

Determine whether this user has (at least one of) the specified role kind(s) in relation to the specified user.

Parameters:

• user – The user that is the target of the role (for which this user has the roles).
• kinds (string from kolibri.auth.constants.role_kinds.*) – The kind (or kinds) of role to check for, as a string or iterable.

Returns:

True if this user has the specified role kind with respect to the target user, otherwise False.

Return type:

bool

is_member_of(coll)

Determine whether this user is a member of the specified Collection.

Parameters:coll – The Collection for which we are checking this user’s membership. Returns:True if this user is a member of the specified Collection, otherwise False. Return type:bool

class kolibri.auth.models.LearnerGroup(id, _morango_dirty_bit, _morango_source_id, _morango_partition, dataset, name, parent, kind, lft, rght, tree_id, level)

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• name (CharField) – Name
• parent_id (TreeForeignKey to ~) – Parent
• kind (CharField) – Kind
• lft (PositiveIntegerField) – Lft
• rght (PositiveIntegerField) – Rght
• tree_id (PositiveIntegerField) – Tree id
• level (PositiveIntegerField) – Level

exception DoesNotExist

exception MultipleObjectsReturned

get_classroom()

Gets the LearnerGroup’s parent Classroom.

Returns:A Classroom instance.

save(*args, **kwargs)

If this is a new node, sets tree fields up before it is inserted into the database, making room in the tree structure as neccessary, defaulting to making the new node the last child of its parent.

It the node’s left and right edge indicators already been set, we take this as indication that the node has already been set up for insertion, so its tree fields are left untouched.

If this is an existing node and its parent has been changed, performs reparenting in the tree structure, defaulting to making the node the last child of its new parent.

In either case, if the node’s class has its order_insertion_by tree option set, the node will be inserted or moved to the appropriate position to maintain ordering by the specified field.

class kolibri.auth.models.Membership(*args, **kwargs)

A FacilityUser can be marked as a member of a Collection through a Membership object. Being a member of a Collection also means being a member of all the Collections above that Collection in the tree (i.e. if you are a member of a LearnerGroup, you are also a member of the Classroom that contains that LearnerGroup, and of the Facility that contains that Classroom).

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• user_id (ForeignKey to ~) – User
• collection_id (TreeForeignKey to ~) – Collection

exception DoesNotExist

exception MultipleObjectsReturned

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.auth.models.Role(*args, **kwargs)

A FacilityUser can have a role for a particular Collection through a Role object, which also stores the “kind” of the Role (currently, one of “admin” or “coach”). Having a role for a Collection also implies having that role for all sub-collections of that Collection (i.e. all the Collections below it in the tree).

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• user_id (ForeignKey to ~) – User
• collection_id (TreeForeignKey to ~) – Collection
• kind (CharField) – Kind

exception DoesNotExist

exception MultipleObjectsReturned

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

User log module

This is a core module found in kolibri/logger.

Concepts and Definitions

Content Interaction Log

This Model provides a record of an interaction with a content item. As such, it should encode the channel that the content was in, and the id of the content. Further, it may be required to encode arbitrary data in a JSON blob that is specific to the particular content type.

As a typical use case, a ContentInteractionLog object might be used to record an interaction with one instance of an exercise (i.e. one question, but possibly multiple attempts within the same session), or a single session of viewing a video.

Finally, these Logs will use MorangoDB to synchronize their data across devices.

Content Summary Log

This Model provides a summary of all interactions of a user with a content item. As such, it should encode the channel that the content was in, and the id of the content. Further, it may be required to encode arbitrary data in a JSON blob that is specific to the particular content type.

As a typical use case, a ContentSummaryLog object might be used to provide summary data about the state of completion of a particular exercise, video, or other content.

When a new InteractionLog is saved, the associated SummaryLog is updated at the same time. This means that the SummaryLog acts as an aggregation layer for the current state of progress for a particular piece of content.

To implement this, a content viewer app would define the aggregation function that summarizes interaction logs into the summary log. While this could happen in the frontend, it would probably be more efficient for this to happen on the server side.

Finally, these Logs will use MorangoDB to synchronize their data across devices - in the case where two summary logs from different devices conflict, then the aggregation logic would be applied across all interaction logs to create a consolidated summary log.

Content Rating Log

This Model provides a record of user feedback on content.

As a typical use case, a ContentRatingLog object might be used to record user feedback data about any content.

Finally, these Logs will use MorangoDB to synchronize their data across devices.

User Session Log

This Model provides a record of an user session in Kolibri. As such, it should encode the channels interacted with, the length of time engaged, and the pages visited.

As a typical use case, a UserSessionLog object might be used to record which pages a user visits, and how long the user is logged on for.

Finally, these Logs will use MorangoDB to synchronize their data across devices.

Implementation Details

Permissions

See Encoding Permission Rules.

Models

This app is intended to provide the core functionality for tracking user engagement with content and Kolibri in general. As such, it is intended to store details of user interactions with content, a summary of those interactions, interactions with the software in general, as well as user feedback on the content and the software.

class kolibri.logger.models.AttemptLog(*args, **kwargs)

This model provides a summary of a user’s engagement within a particular interaction with an item/question in an assessment

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• item (CharField) – Item
• start_timestamp (DateTimeTzField) – Start timestamp
• end_timestamp (DateTimeTzField) – End timestamp
• completion_timestamp (DateTimeTzField) – Completion timestamp
• time_spent (FloatField) – (in seconds)
• complete (BooleanField) – Complete
• correct (FloatField) – Correct
• hinted (BooleanField) – Hinted
• answer (JSONField) – Answer
• simple_answer (CharField) – Simple answer
• interaction_history (JSONField) – Interaction history
• user_id (ForeignKey to ~) – User
• error (BooleanField) – Error
• masterylog_id (ForeignKey to ~) – Masterylog
• sessionlog_id (ForeignKey to ~) – Sessionlog

exception DoesNotExist

exception MultipleObjectsReturned

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.BaseAttemptLog(*args, **kwargs)

This is an abstract model that provides a summary of a user’s engagement within a particular interaction with an item/question in an assessment

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• item (CharField) – Item
• start_timestamp (DateTimeTzField) – Start timestamp
• end_timestamp (DateTimeTzField) – End timestamp
• completion_timestamp (DateTimeTzField) – Completion timestamp
• time_spent (FloatField) – (in seconds)
• complete (BooleanField) – Complete
• correct (FloatField) – Correct
• hinted (BooleanField) – Hinted
• answer (JSONField) – Answer
• simple_answer (CharField) – Simple answer
• interaction_history (JSONField) – Interaction history
• user_id (ForeignKey to ~) – User
• error (BooleanField) – Error

class kolibri.logger.models.BaseLogModel(*args, **kwargs)

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.BaseLogQuerySet(model=None, query=None, using=None, hints=None)

filter_by_content_ids(content_ids, content_id_lookup='content_id')

Filter a set of logs by content_id, using content_ids from the provided list or queryset.

filter_by_topic(topic, content_id_lookup='content_id')

Filter a set of logs by content_id, using content_ids from all descendants of specified topic.

class kolibri.logger.models.ContentSessionLog(*args, **kwargs)

This model provides a record of interactions with a content item within a single visit to that content page.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• user_id (ForeignKey to ~) – User
• content_id (UUIDField) – Content id
• channel_id (UUIDField) – Channel id
• start_timestamp (DateTimeTzField) – Start timestamp
• end_timestamp (DateTimeTzField) – End timestamp
• time_spent (FloatField) – (in seconds)
• progress (FloatField) – Progress
• kind (CharField) – Kind
• extra_fields (JSONField) – Extra fields

exception DoesNotExist

exception MultipleObjectsReturned

save(*args, **kwargs)

Saves the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

class kolibri.logger.models.ContentSummaryLog(*args, **kwargs)

This model provides a summary of all interactions a user has had with a content item.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• user_id (ForeignKey to ~) – User
• content_id (UUIDField) – Content id
• channel_id (UUIDField) – Channel id
• start_timestamp (DateTimeTzField) – Start timestamp
• end_timestamp (DateTimeTzField) – End timestamp
• completion_timestamp (DateTimeTzField) – Completion timestamp
• time_spent (FloatField) – (in seconds)
• progress (FloatField) – Progress
• kind (CharField) – Kind
• extra_fields (JSONField) – Extra fields

exception DoesNotExist

exception MultipleObjectsReturned

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

save(*args, **kwargs)

Saves the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

class kolibri.logger.models.ExamAttemptLog(*args, **kwargs)

This model provides a summary of a user’s engagement within a particular interaction with an item/question in an exam

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• item (CharField) – Item
• start_timestamp (DateTimeTzField) – Start timestamp
• end_timestamp (DateTimeTzField) – End timestamp
• completion_timestamp (DateTimeTzField) – Completion timestamp
• time_spent (FloatField) – (in seconds)
• complete (BooleanField) – Complete
• correct (FloatField) – Correct
• hinted (BooleanField) – Hinted
• answer (JSONField) – Answer
• simple_answer (CharField) – Simple answer
• interaction_history (JSONField) – Interaction history
• user_id (ForeignKey to ~) – User
• error (BooleanField) – Error
• examlog_id (ForeignKey to ~) – Examlog
• content_id (UUIDField) – Content id
• channel_id (UUIDField) – Channel id

exception DoesNotExist

exception MultipleObjectsReturned

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.ExamLog(*args, **kwargs)

This model provides a summary of a user’s interaction with a particular exam, and serves as an aggregation point for individual attempts on that exam.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• exam_id (ForeignKey to ~) – Exam
• user_id (ForeignKey to ~) – User
• closed (BooleanField) – Closed
• completion_timestamp (DateTimeTzField) – Completion timestamp

exception DoesNotExist

exception MultipleObjectsReturned

calculate_partition()

Should return a string specifying this model instance’s partition, using self.ID_PLACEHOLDER in place of its own ID, if needed.

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

class kolibri.logger.models.MasteryLog(*args, **kwargs)

This model provides a summary of a user’s engagement with an assessment within a mastery level

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• user_id (ForeignKey to ~) – User
• summarylog_id (ForeignKey to ~) – Summarylog
• mastery_criterion (JSONField) – Mastery criterion
• start_timestamp (DateTimeTzField) – Start timestamp
• end_timestamp (DateTimeTzField) – End timestamp
• completion_timestamp (DateTimeTzField) – Completion timestamp
• mastery_level (IntegerField) – Mastery level
• complete (BooleanField) – Complete

exception DoesNotExist

exception MultipleObjectsReturned

calculate_source_id()

Should return a string that uniquely defines the model instance or None for a random uuid.

infer_dataset(*args, **kwargs)

This method is used by ensure_dataset to “infer” which dataset should be associated with this instance. It should be overridden in any subclass of AbstractFacilityDataModel, to define a model-specific inference.

class kolibri.logger.models.UserSessionLog(*args, **kwargs)

This model provides a record of a user session in Kolibri.

Parameters:

• id (UUIDField) – Id
• _morango_dirty_bit (BooleanField) – morango dirty bit
• _morango_source_id (CharField) – morango source id
• _morango_partition (CharField) – morango partition
• dataset_id (ForeignKey to ~) – Dataset
• user_id (ForeignKey to ~) – User
• channels (TextField) – Channels
• start_timestamp (DateTimeTzField) – Start timestamp
• last_interaction_timestamp (DateTimeTzField) – Last interaction timestamp
• pages (TextField) – Pages

exception DoesNotExist

exception MultipleObjectsReturned

classmethod update_log(user)

Update the current UserSessionLog for a particular user.

Kolibri plugin architecture

The behavior of Kolibri can be extended using plugins. The following is a guide to developing plugins.

Enabling and disabling plugins

Non-core plugins can be enabled or disabled using the kolibri plugin command. See ../user/cli.

How plugins work

From a user’s perspective, plugins are enabled and disabled through the command line interface or through a UI. Users can also configure a plugin’s behavior through the main Kolibri interface.

Note

We have not yet written a configuration API, for now just make sure configuration-related variables are kept in a central location of your plugin.

It’s up to the plugin to provide configuration Form classes and register them.

We should aim for a configuration style in which data can be pre-seeded, dumped and exported easily.

From a developer’s perspective, plugins are Django applications listed in INSTALLED_APPS and are initialized once when the server starts, mean at the load time of the django project, i.e. Kolibri.

Loading a plugin

In general, a plugin should never modify internals of Kolibri or other plugins without using the hooks API or normal conventional Django scenarios.

Note

Each app in INSTALLED_APPS is searched for the special kolibri_plugin module.

Everything that a plugin does is expected to be defined through <myapp>/kolibri_plugin.py.

Kolibri Hooks API

What are hooks

Hooks are classes that define something that happens at one or more places where the hook is looked for and applied. It means that you can “hook into a component” in Kolibri and have it do a predefined and parameterized thing. For instance, Kolibri could ask all its plugins who wants to add something to the user settings panel, and its then up to the plugins to inherit from that specific hook and feed back the parameters that the hook definition expects.

The consequences of a hook being applied can happen anywhere in Kolibri. Each hook is defined through a class inheriting from KolibriHook. But how the inheritor of that class deals with plugins using it, is entirely up to each specific implementation and can be applied in templates, views, middleware - basically everywhere!

That’s why you should consult the class definition and documentation of the hook you are adding plugin functionality with.

We have two different types of hooks:

Abstract hooks

Are definitions of hooks that are implemented by implementing hooks.

Registered hooks

Are concrete hooks that inherit from abstract hooks, thus embodying the definitions of the abstract hook into a specific case.

So what’s “a hook”?

Simply referring to “a hook” is okay, it can be ambiguous on purpose. For instance, in the example, we talk about “a navigation hook”. So we both mean the abstract definition of the navigation hook and everything that is registered for the navigation.

Where can I find hooks?

All Kolibri core applications and plugins alike should by convention define their abstract hooks inside <myapp>/hooks.py. Thus, to see which hooks a Kolibri component exposes, you can refer to its hooks module.

Note

Defining abstract hooks in <myapp>/hooks.py isn’t mandatory, but loading a concrete hook in <myapp>/kolibri_plugin.py is.

Warning

Do not define abstract and registered hooks in the same module. Or to put it in other words: Never put registered hooks in <myapp>/hooks.py. The non-abstract hooks should not be loaded unintentionally in case your application is not loaded but only used to import an abstract definition by an external component!

In which order are hooks used/applied?

This is entirely up to the registering class. By default, hooks are applied in the same order that the registered hook gets registered! This most likely means the order in which kolibri_plugin is loaded => the order in which the app is listed in INSTALLED_APPS

An example of a plugin using a hook

Note

The example shows a NavigationHook which is simplified for the sake of readability. The actual implementation in Kolibri will defer.

Example implementation

Here is an example of how to use a hook in myplugin.kolibri_plugin.py:

from django.db.models

# This is where the actual abstract hook is defined
from kolibri.core.hooks import NavigationHook

# By inheriting NavigationHook, we tell that we are going to want our
# plugin to be part of the hook's activities with the specified attributes.
# We only define one navigation item, but defining another one is as simple
# as adding another class definition.
class MyPluginNavigationItem(NavigationHook):

    label = _("My Plugin")
    url = reverse_lazy("kolibri:my_plugin:index")

And here is the definition of that hook in kolibri.core.hooks:

from kolibri.plugins.hooks import KolibriHook


class NavigationHook(KolibriHook):
    """
    Extend this hook to define a new navigation item
    """

    #: A string label for the menu item
    label = "Untitled"

    #: A string or lazy proxy for the url
    url = "/"

    @classmethod
    def get_menu(cls):
        menu = {}
        for hook in self.registered_hooks:
            menu[hook.label] = url
        return menu

    class Meta:

        abstract = True

Usage of the hook

Inside our templates, we load a template tag from navigation_tags, and this template tag definition looks like this:

from kolibri.core.hooks import NavigationHook

@register.assignment_tag()
def kolibri_main_navigation():

    for item in NavigationHook().get_menu():
        yield item

{% load kolibri_tags %}

<ul>
{% for menu_item in kolibri_main_navigation %}
    <li><a href="{{ menu_item.url }}">{{ menu_item.label }}</a></li>
{% endfor %}
</ul>

Warning

Do not load registered hook classes outside of a plugin’s kolibri_plugin. Either define them there directly or import the modules that define them. Hook classes should all be seen at load time, and placing that logic in kolibri_plugin guarantees that things are registered correctly.

Other stuff you can do with plugins

Plugins can implement Javascript code as a Kolibri module that can be used in the frontend as a plugin to the core Kolibri Javascript code. Each of these Javascript plugins are defined in the kolibri_plugin.py file by subclassing the KolibriFrontEndPluginBase class to define each frontend Kolibri module. This defines the base Javascript file that defines the Kolibri module. In addition, this Plugin object within the app will automatically add these Kolibri modules to an internal frontend asset registry for loading in the front end. For more information on developing frontend code for Kolibri please see Front-end architecture.

Plugins can be standalone Django apps in their own right, meaning they can define templates, models, new urls, and views just like any other app. However the API for all of this hasn’t yet been determined… Coming soon!

Core plugin example

View the source to learn more!

class kolibri.core.kolibri_plugin.KolibriCore

The most minimal plugin possible. Because it’s in the core, it doesn’t define enable or disable. Those methods should never be called for this plugin.

Builds

Distribution build pipeline

The Kolibri Package build pipeline looks like this:

                       Git master branch
                               |
                               |
                              / \
                             /   \
Python dist, online dependencies  \
   `python setup.py bdist_wheel`   \
               /                    \
              /                Python dist, bundled dependencies
       Upload to PyPi        `python setup.py bdist_wheel --static`
      Installable with                 \
    `pip install kolibri`               \
                                   Upload to PyPi
                                   Installable with
                             `pip install kolibri-static`
                               /            |          \
                              /             |           \
                        Windows            OSX        Debian
                       installer         installer   installer

Make targets

To build both the slim Kolibri and the one with bundled dependencies, simply run make dist. The .whl files will now be available in dist/*whl and you can install them with pip install dist/filename.whl.

Front-end build pipeline

Asset pipelining is done using Webpack - this allows the use of require to import modules - as such all written code should be highly modular, individual files should be responsible for exporting a single function or object.

There are two distinct entities that control this behaviour - a Kolibri Hook on the Python side, which manages the registration of the frontend code within Django (and also facilitates building of that code into compiled assets with Webpack) and a Kolibri Module (a subclass of KolibriModule) on the JavaScript side (see Front-end architecture).

Kolibri has a system for synchronously and asynchronously loading these bundled JavaScript modules which is mediated by a small core JavaScript app, kolibriGlobal. Kolibri Modules define to which events they subscribe, and asynchronously registered Kolibri Modules are loaded by kolibriGlobal only when those events are triggered. For example if the Video Viewer’s Kolibri Module subscribes to the content_loaded:video event, then when that event is triggered on kolibriGlobal it will asynchronously load the Video Viewer module and re-trigger the content_loaded:video event on the object the module returns.

Synchronous and asynchronous loading is defined by the template tag used to import the JavaScript for the Kolibri Module into the Django template. Synchronous loading merely inserts the JavaScript and CSS for the Kolibri Module directly into the Django template, meaning it is executed at page load.

This can be achieved in two ways using tags defined in kolibri/core/webpack/templatetags/webpack_tags.py.

The first way is simply by using the webpack_asset template tag.

The second way is if a Kolibri Module needs to load in the template defined by another plugin or a core part of Kolibri, a template tag and hook can be defined to register that Kolibri Module’s assets to be loaded on that page. An example of this is found in the base.html template using the webpack_base_assets tag.

This relies on the following function to collect all registered Kolibri Modules and load them synchronously: kolibri.core.webpack.utils.webpack_asset_render

Asynchronous loading can also, analogously, be done in two ways. Asynchronous loading registers a Kolibri Module against kolibriGlobal on the frontend at page load, but does not load, or execute any of the code until the events that the Kolibri Module specifies are triggered. When these are triggered, the kolibriGlobal will load the Kolibri Module and pass on any callbacks once it has initialized. Asynchronous loading can be done either explicitly with a template tag that directly imports a single Kolibri Module using webpack_base_async_assets.

Upgrading

Warning

These instructions are under development

Upgrade paths

Kolibri can be automatically upgraded forwards. For instance, you can upgrade from 0.1->0.2 and 0.1->0.7. We test all upgrade paths, but we also caution that the more versions that you skip, the higher the risks will be that something isn’t working as expected.

That’s why we also support Downgrading.

Every time Kolibri is upgraded, it will automatically migrate your database and create a backup before doing so.

Note

Always upgrade as often as possible. If you are responsible for deployments at different sites, you should consider a strategy for keeping software and contents updated.

Downgrading

To downgrade you need to do two steps:

1. If you have been using the latest version and want to store data, make sure to create a backup before continuing: kalite manage dbbackup
2. Install the older version on top of the new version using the same installation type.
3. Restore the latest Database backup.

When you upgrade Kolibri, the database is changed to match the latest version of Kolibri, however these changes cannot be unmade. That’s why you need to restore the database from a backup.

Database backup

While upgrading, Kolibri will automatically generate a backup of the database before making any changes. This guarantees that in case the upgrade causes problems, you can downgrade and restore the backup.

Backups

Kolibri stores database backups in ~/.kolibri/backups. The dump files created contain SQL statements to be run by SQLite3. You can re-instate a dump by using the special dbrestore command.

Restoring from backup

Warning

Restoring from backup will overwrite the current database, so store a backup in case you have data you want to preserve!

To restore from the latest available backup, run the following from command line:

$ kolibri manage dbrestore --latest

To restore from a specific backup file:

$ kolibri manage dbrestore /path/to/db-backup.dump

Release Notes

Changes are ordered reverse-chronologically.

0.10.3

Internationalization and localization

• Added Mexican Spanish (es_MX) and Bulgarian (bg)

Fixed

• Upgrade issue upon username conflict between device owner and facility user
• Channel import listing of USB devices when non-US locale
• Counts for coach-specific content would in some cases be wrongly displayed

See a more detailed list of changes on Github

0.10.2

• Performance improvements and bug fixes for content import
• Exam creation optimizations

See a more detailed list of changes on Github

0.10.1

• Bug fix release
• Several smaller UI fixes
• Fixes for SSL issues on low-spec devices / unstable connectivity
• Compatibility fixes for older system libraries

See a more detailed list of changes on Github

0.10.0

• Support for coach-specific content
• Content import/export is more reliable and easier to use
• Search has improved results and handles duplicate items
• Display of answer history in learner exercises is improved
• Login page is more responsive
• Windows-specific improvements and bug fixes
• New Kolibri configuration file
• Overall improved performance
• Auto-play videos
• Various improvements to PDF renderer
• Command to migrate content directory location
• Languages: English, Arabic, Bengali, Chinyanja, Farsi, French, Hindi, Kannada, Marathi, Burmese, Portuguese (Brazilian), Spanish, Swahili, Tamil, Telugu, Urdu, Yoruba, and Zulu

See a more detailed list of changes on Github.

0.9.3

• Compressed database upload
• Various bug fixes

See a more detailed list of changes on Github.

0.9.2

• Various bug fixes
• Languages: English, Arabic, Bengali, Chinyanja, Farsi, French, Hindi, Marathi, Portuguese (Brazilian), Spanish, Swahili, Tamil, Telugu, Urdu, Yoruba, and Zulu

See a more detailed list of changes on Github.

0.9.1

• Fixed regression that caused very slow imports of large channels
• Adds new ‘import users’ command to the command-line
• Various consistency and layout updates
• Exercises with an error no longer count as ‘correct’
• Fixed issue with password-less sign-on
• Fixed issue with editing lessons
• Various other fixes
• Languages: English, Arabic, Chinyanja, Farsi, French, Hindi, Marathi, Portuguese (Brazilian), Spanish, Swahili, Tamil, Telugu, and Urdu

See a more detailed list of changes on Github.

0.9.0

• Consistent usage of ‘coach’ terminology
• Added class-scoped coaches
• Support for multi-facility selection on login
• Cross-channel exams
• Show correct and submitted answers in exam reports
• Added learner exam reports
• Various bug fixes in exam creation and reports
• Various bug fixes in coach reports
• Fixed logging on Windows
• Added ability for coaches to make copies of exams
• Added icon next to language-switching functionality
• Languages: English, Arabic, Farsi, French, Hindi, Spanish, Swahili, and Urdu

See a more detailed list of changes on Github.

0.8.0

• Added support for assigning content using ‘Lessons’
• Updated default landing pages in Learn and Coach
• Added ‘change password’ functionality to ‘Profile’ page
• Updates to text consistency
• Languages: English, Spanish, Arabic, Farsi, Urdu, French, Haitian Creole, and Burmese
• Various bug fixes

See a more detailed list of changes on Github.

0.7.2

• Fix issue with importing large channels on Windows
• Fix issue that prevented importing topic thumbnail files

0.7.1

• Improvements and fixes to installers including Windows & Debian
• Updated documentation

0.7.0

• Completed RTL language support
• Languages: English, Spanish, Arabic, Farsi, Swahili, Urdu, and French
• Support for Python 3.6
• Split user and developer documentation
• Improved lost-connection and session timeout handling
• Added ‘device info’ administrator page
• Content search integration with Studio
• Granular content import and export

0.6.2

• Consistent ordering of channels in learner views

0.6.1

• Many mobile-friendly updates across the app
• Update French, Portuguese, and Swahili translations
• Upgraded Windows installer

0.6.0

• Cross-channel searching and browsing
• Improved device onboarding experience
• Improved device permissions experience (deprecated ‘device owner’, added ‘superuser’ flag and import permission)
• Various channel import/export experience and stability improvements
• Responsive login page
• Dynamic language switching
• Work on integrated living style guide
• Added beta support for right-to-left languages
• Improved handling of locale codes
• Added support for frontend translation outside of Vue components
• Added an open-source ‘code of conduct’ for contributors
• By default run PEX file in foreground on MacOS
• Crypto optimizations from C extensions
• Deprecated support for HTML in translation strings
• Hide thumbnails from content ‘download’ button
• Automatic database backup during upgrades. #2365
• … and many other updates and fixes

0.5.3

• Release timeout bug fix from 0.4.8

0.5.2

• Release bug fix from 0.4.7

0.5.1

• Python dependencies: Only bundle, do not install dependencies in system env #2299
• Beta Android support
• Fix ‘importchannel’ command #2082
• Small translation improvements for Spanish, French, Hindi, and Swahili

0.5.0

• Update all user logging related timestamps to a custom datetime field that includes timezone info
• Added daemon mode (system service) to run kolibri start in background (default!) #1548
• Implemented kolibri stop and kolibri status #1548
• Newly imported channels are given a ‘last_updated’ timestamp
• Add progress annotation for topics, lazily loaded to increase page load performance
• Add API endpoint for getting number and total size of files in a channel
• Migrate all JS linting to prettier rather than eslint
• Merge audio_mp3_render and video_mp4_render plugins into one single media_player plugin
• KOLIBRI_LISTEN_PORT environment variable for specifying a default for the –port option #1724

0.4.9

• User experience improvements for session timeout

0.4.8

• Prevent session timeout if user is still active
• Fix exam completion timestamp bug
• Prevent exercise attempt logging crosstalk bug
• Update Hindi translations

0.4.7

• Fix bug that made updating existing Django models from the frontend impossible

0.4.6

• Fix various exam and progress tracking issues
• Add automatic sign-out when browser is closed
• Fix search issue
• Learner UI updates
• Updated Hindi translations

0.4.5

• Frontend and backend changes to increase performance of the Kolibri application under heavy load
• Fix bug in frontend simplified login code

0.4.4

• Fix for Python 3 compatibility in Whl, Windows and Pex builds #1797
• Adds Mexican Spanish as an interface language
• Upgrades django-q for bug fixes

0.4.3

• Speed improvements for content recommendation #1798

0.4.2

• Fixes for morango database migrations

0.4.1

• Makes usernames for login case insensitive #1733
• Fixes various issues with exercise rendering #1757
• Removes wrong CLI usage instructions #1742

0.4.0

• Class and group management
• Learner reports #1464
• Performance optimizations #1499
• Anonymous exercises fixed #1466
• Integrated Morango, to prep for data syncing (will require fresh database)
• Adds Simplified Login support as a configurable facility flag

0.3.3

• Turns video captions on by default

0.3.2

• Updated translations for Portuguese and Kiswahili in exercises.
• Updated Spanish translations

0.3.1

• Portuguese and Kaswihili updates
• Windows fixes (mimetypes and modified time)
• VF sidebar translations

0.3.0

• Add support for nested URL structures in API Resource layer
• Add Spanish and Swahili translations
• Improve pipeline for translating plugins
• Add search back in
• Content Renderers use explicit new API rather than event-based loading

0.2.0

• Add authentication for tasks API
• Temporarily remove ‘search’ functionality
• Rename ‘Learn/Explore’ to ‘Recommended/Topics’
• Add JS-based ‘responsive mixin’ as alternative to media queries
• Replace jeet grids with pure.css grids
• Begin using some keen-ui components
• Update primary layout and navigation
• New log-in page
• User sign-up and profile-editing functionality
• Versioning based on git tags
• Client heartbeat for usage tracking
• Allow plugins to override core components
• Wrap all user-facing strings for I18N
• Log filtering based on users and collections
• Improved docs
• Pin dependencies with Yarn
• ES2015 transpilation now Bublé instead of Babel
• Webpack build process compatible with plugins outside the kolibri directory
• Vue2 refactor
• HTML5 app renderer

0.1.1

• SVG inlining
• Exercise completion visualization
• Perseus exercise renderer
• Coach reports

0.1.0 - MVP

• Improved documentation
• Conditional (cancelable) JS promises
• Asset bundling performance improvements
• Endpoint indexing into zip files
• Case-insensitive usernames
• Make plugins more self-contained
• Client-side router bug fixes
• Resource layer smart cache busting
• Loading ‘spinner’
• Make modals accessible
• Fuzzy searching
• Usage data export
• Drive enumeration
• Content interaction logging
• I18N string extraction
• Channel switching bug fixes
• Modal popups
• A11Y updates
• Tab focus highlights
• Learn app styling changes
• User management UI
• Task management
• Content import/export
• Session state and login widget
• Channel switching
• Setup wizard plugin
• Documentation updates
• Content downloading

0.0.1 - MMVP

• Page titles
• Javascript logging module
• Responsiveness updates
• A11Y updates
• Cherrypy server
• Vuex integration
• Stylus/Jeet-based grids
• Support for multiple content DBs
• API resource retrieval and caching
• Content recommendation endpoints
• Client-side routing
• Content search
• Video, Document, and MP3 content renderers
• Initial VueIntl integration
• User management API
• Vue.js integration
• Learn app and content browsing
• Content endpoints
• Automatic inclusion of requirements in a static build
• Django JS Reverse with urls representation in kolibriGlobal object
• Python plugin API with hooks
• Webpack build pipeline, including linting
• Authentication, authorization, permissions
• Users, Collections, and Roles