Practice test · Python and local AI

A test that follows you, on every lesson of the syllabus

This page tests what the lessons teach: every lesson in the Python and local AI syllabus, and every lesson that Start Python sends you to. Each lesson is one topic. The page asks fifteen questions at a time. It keeps a score for each topic, finds the three where you are weakest, and asks about those. After each answer it says why, and gives you a link to the part of the lesson that the question comes from.

Do a lesson before you are tested on it. Under the test you choose the units of the syllabus that you have done, and the test stays inside them.

You need nothing but this page. Your scores are kept in this browser only, and nothing is sent anywhere.

Loading the questions…

How the test chooses its questions

Nothing in the test is AI. It is arithmetic on the answers you gave, and all of it is on this page.

How the adaptive learning cycle works

Each round is one turn of this cycle. The test picks questions from your weakest lessons, records your answers, moves your score for each lesson, and picks harder or easier questions the next time.

  1. Rank the lessons. Each lesson is a topic, and each topic in the units you chose gets a priority. The weakest float to the top.
  2. Build the round. Your three weakest lessons get five questions each. For each one the test picks the questions nearest to your level.
  3. Answer and learn. You answer each question. Then you see why, and a link to that part of the lesson.
  4. Move θ. The Elo formula moves your score for that lesson up for a right answer and down for a wrong one.
  5. Show the results. Your score for the round, a bar for each lesson, your mastery, and the lessons for next time.
  6. Next round. The new priorities go back to step 1. A weak lesson keeps getting questions until you master it.

The math behind it

The test uses an Elo rating to follow how well you know each lesson, and it uses that score to choose the questions of your next round.

The mastery curve (a sigmoid)

0% 50% 100% Weak Middle Strong θ = 0

Left to right is the score, θ, from −3 to +3. Bottom to top is mastery as a percentage. Every lesson has its own θ. It starts at 0, which is 50%, and your answers move it up or down.

Three steps of an Elo update

  1. You get one wrong. θ drops. For a question at your own level the drop is about 0.3 during your first ten answers on that lesson, and about 0.15 after that. It is never more than 0.6.
  2. θ drops, so mastery drops, so the priority rises. The weakness is squared, so a weak lesson rises fast.
  3. A higher priority puts the lesson in your next round. You practise it more, until its mastery climbs back.

The formulas

Elo update: new θ = old θ + K × (right - expected) right is 1 or 0 Expected: 1 / (1 + e^(-(θ - difficulty))) Mastery: 1 / (1 + e^(-θ)) Priority: (1 - mastery)² × untouched × 1 / (1 + 0.1 × attempts)

K is 0.6 for your first ten answers on a lesson and 0.3 after that, so the score moves fast at first and then settles. untouched is 1.3 for a lesson you have never been asked about, and 1 otherwise. Each question has a difficulty from −1 (easy) to 1 (hard).

This works like the Elo ratings of chess. You do not play another person: you play the difficulty of each question. A right answer to a hard question earns more θ than a right answer to an easy one.

How adaptive practice works

Each time you press Begin, the test builds a round of fifteen questions for you. The questions are not drawn by chance. The test finds your three weakest lessons and gives each one five questions, so that you practise where you need it.

Your profile

The test keeps a score, θ, for every lesson. Every θ starts at 0, which is 50% mastery. A right answer moves it up and a wrong answer moves it down, by the Elo formula above. Your profile is saved in this browser, so it is still there the next time you open the page. Start from zero, on the results screen, clears it.

Building a round

The test ranks the lessons in the units you chose, takes the three with the highest priority, and picks five questions for each: the ones whose difficulty is nearest to your θ for that lesson. A question you have seen is left out for the next three rounds, unless the lesson has no others left.

Kinds of question

Most questions give you four answers to choose from, and the four are shown in a new order each time. A few ask you to type one word, such as the name of a command. Every question belongs to one lesson and has a difficulty, so the score for that lesson can follow you.

The question pool

Loading…

Where the questions come from