_______ __ ________ __
| \ | \ | \ | \
| βββββββ\ ______ ______ \ββ_______ \ββββββββ ______ \ββ _______
| ββ__/ ββ/ \ | \| \ \ | ββ / \| \/ \
| ββ ββ ββββββ\ \ββββββ\ ββ βββββββ\ | ββ | ββββββ\ ββ βββββββ
| βββββββ\ ββ \ββ/ ββ ββ ββ | ββ | ββ | ββ \ββ ββ\ββ \
| ββ__/ ββ ββ | βββββββ ββ ββ | ββ | ββ | ββ | ββ_\ββββββ\
| ββ ββ ββ \ββ ββ ββ ββ | ββ | ββ | ββ | ββ ββ
\βββββββ \ββ \βββββββ\ββ\ββ \ββ \ββ \ββ \ββ\βββββββ
Welcome to the BrainTris Lab!
BrainTris is an in-browser Tetris and reinforcement-learning laboratory. You can play manually,
inspect the configured reward heuristic, train a Deep Q-Network, or watch a trained network
play. The project is designed for experimenting with
Q-Learning
and observing how reward design and training settings change behavior.
Start exploring
To watch the AI, select Play AI. No download or training is needed:
it uses the network already loaded in the application, then tries this browser's saved model
if necessary. If no usable browser model is available, it automatically loads
/brain.bbt from the website.
To build your own network, choose Train →
Start New, review the options, and select
Apply. Use Continue to resume
saved training. Export a complete save before starting over if you want to keep your experiment.
What the network sees
Each candidate placement is encoded as three 22 Γ 10 visual channels:
- the fixed stack after placing the piece and clearing completed lines;
- column-shadow holes below occupied cells;
- the next tetromino, encoded by shape or piece ID according to the selected option.
Alongside these images, two separate scalar inputs describe the placement: cleared lines divided by four,
and whether the game ends. These values join the visual features before the final value prediction.
The 22 rows include two hidden spawn rows above the 20-row playing area. Aggregate height and
bumpiness are used for rewards and diagnostics rather than supplied as extra network inputs.
Tetris
Complete horizontal lines to remove them and keep the stack below the top of the board. Clearing
four lines at once is counted as a Tetris. Optional penalty rows and extra tetrominoes can make
the environment harder during experiments.
Game and Learning Modes
The application offers four modes:
- Human: manual Tetris;
- Play Heuristic: the configured reward function chooses moves;
- Train: collects experience and updates the neural network;
- Play AI: a trained network selects moves without exploration.
Human play
Play with the keyboard. The Human and AI play surfaces are rendered by PixiJS.
- ← / →: move;
- ↓: soft drop or accelerate AI playback;
- ↑ or SPACE: rotate;
- P: pause or resume;
- M: toggle music.
Lines and placed pieces contribute to the score. The level increases as lines are cleared, and
the game ends when a piece cannot spawn or locks too high in the spawn area.
Play Heuristic
Plays without loading or querying a neural network. It evaluates legal placements with the
currently configured reward function and a short look-ahead, making it useful for inspecting
reward behavior before or during training experiments.
Network training (Train mode)
Training enumerates legal placements, stores encoded experiences in replay memory, and performs
periodic DQN updates. Training intentionally runs without rendering the game board; the DOM
training panel and charts provide feedback instead.
The Train submenu contains:
-
Continue: resume the saved training when one exists;
-
Start New: if saved training exists, confirm its deletion,
then open Training Options; without saved training it opens the options directly. The new
training starts only when Apply is selected;
-
Save: export either a complete training archive or an
inference-only network;
-
Save & Stop: finish the same export first, then run the
stop training without resuming it between the two operations. If export fails, training is not stopped;
-
Load: restore a .bbt file
selected from disk or dropped into the load area;
- Reset: reset the current training state after confirmation.
Save files
A complete training archive contains the online and target networks, optimizer state, training
counters and options, the current game, charts, piece-generator state, and replay-buffer data.
Its filename starts with the training ID and also includes a compact
six-digit save time and the rounded moving-average rows.
Inference-only export contains the network for playback, not a complete training checkpoint.
Both use the .bbt extension. Publishing an inference export as
brain.bbt in the site root supplies the automatic Play AI model.
Browser saves keep the model and training checkpoint locally, but omit replay samples:
replay memory starts empty after a browser reload. Use a complete file export to preserve replay
as well. Browser storage belongs to this browser profile and website address; clearing site data
removes it. Closing the tab can only recover the last completed save, so save explicitly before leaving.
Play AI
Play AI uses an already loaded network, otherwise tries the model saved in this browser, then
/brain.bbt from the site root. It disables exploration and learning and
lets the network choose the highest-valued legal placement. Holding the down arrow accelerates
playback and suppresses nonessential effects.
A successfully fetched site model is also cached in the browser when storage is available.
If loading fails, the application shows an error. Older networks with a different input layout
are incompatible with the current three-channel and two-scalar model: use a compatible save or
start a new training run. Watching AI play does not resume training or rebuild replay memory.
Stop
Stops the active mode and returns to idle. Stopping training saves its browser state and model;
file export remains a separate action.
Other menu actions
-
Options edits reward, learning, gameplay, rendering, and
monitoring settings.
-
Delete all data removes locally saved training state,
charts, and browser model data, then reloads the application. It does not delete the site's
root brain.bbt.
- The theme selector changes the application color theme.
Training Options
Options edits the configuration used by the next training run
or updates the active training configuration. The question-mark buttons beside individual
fields provide short field-level explanations.
Reward and Punishment
Positive values reward a result and negative values penalize it. The active reward inputs are:
- Completed Lines: reward weight for clearing lines;
-
Lines Cleared Exponent: makes multi-line clears scale
nonlinearly;
-
Aggregate Height: reward or penalty based on the sum of
all ten column heights;
- Holes: penalty for covered empty cells;
- Bumpiness: penalty for differences between adjacent column heights;
-
Penalty Row: event reward or penalty when a penalty row is
inserted;
- Game End Penalty: terminal penalty when the game ends.
Learning
-
Target Update selects hard or soft synchronization.
-
Target Update Frequency controls periodic hard copies.
-
Target Update Tau controls soft Polyak updates.
-
Use PER (Prioritized Replay) enables prioritized sampling
and importance-sampling weights.
- Batch Size (max 512) controls samples per replay update.
-
Replay Frequency (steps) controls environment steps between
training updates.
Statistical Moving Average Windows
The short and long windows drive Average Rows smoothing and the displayed learning trend. They
are observational statistics, not additional neural-network inputs.
- Moving Avg. Window: short-term window;
- Long-Term Window: comparison window for the trend.
Gameplay, Rendering, and Monitoring
Piece Seed fixes the training piece sequence across games.
Use the same nonzero integer (1β4294967295) and the same Extra Tetrominoes setting to compare runs.
0 chooses a random seed once; the effective seed appears in Options and is saved with the generator state.
Full saves resume that stream. Set the seed before a new training run; it cannot be edited during training.
This controls pieces, while network initialization, exploration, replay sampling and GPU execution remain stochastic.
-
Penalty Rows: enables periodic bottom-row insertion;
-
Extra Tetrominoes: adds non-classic pieces;
- Block Render Style: changes PixiJS block appearance;
-
Stability Panel: shows the optional extended replay and
training diagnostics.
Operations
- Reset to Defaults: restore the form's startup defaults;
shown only when editable values differ from those defaults. Select Apply to commit them;
- Apply: validate and apply the displayed values;
- Cancel: close without applying the displayed changes.
Training Panel and Statistics
Training uses a monitoring panel instead of rendering the game board. Updates are intentionally
sampled at different cadences so charts and diagnostics do not dominate training time.
Network topology
Shows the active visual input shape and two scalar inputs, convolutional branches, compression
stage, flattened feature count, and dense layers.
AI Vision
Periodically displays a simplified view of the candidate board, highlighting the stack, placed
piece, and holes. It is a visual aid, not a display of every network input or every move.
Statistics
- Average Rows and trend compare the configured moving windows;
- Games, current lines, total pieces, and total lines show workload;
- Tetrises shows count and percentage of four-line clears;
- Max Lines and Max Level show records;
-
Env. upd. / s and
Train / s separate environment throughput from replay
updates.
Training Data
Shows the latest batch's terminal ratio, reward mean, target mean, TD mean, predicted-next mean,
replay-buffer size, beta, epsilon, gamma, and learning rate. These are raw diagnostics for the
current replay batch, not a single overall quality score.
Charts
-
Average Rows / Tetris %: smoothed rows and four-line-clear
percentage on independently scaled axes; the Tetris percentage uses a tightly padded range
for its complete visible history without rescaling Average Rows;
- Q-Value: selected action-value trend;
- Loss: replay loss trend;
-
Gradient: gradient norm with the configured clipping
threshold as reference.
Stability panel
When enabled, adds Done Ratio, R Mean, Targets Mean, and TD Mean charts plus measured replay/PER
statistics such as buffer fill, rolling coverage, sample age, effective sample size, priority
concentration, TD clipping, and importance-weight range. It reports measurements rather than an
automatic health grade or tuning recommendation.
Reward options
Shows the effective line, exponent, aggregate-height, holes, bumpiness, penalty-row, and
game-end weights used by the active run.
Training Methods Used
The active implementation is a value-based DQN over complete legal placements. Its main
components are:
- a hybrid encoder with three visual channels, two placement-event scalars, and one Q-value output;
- encoded experience replay, optionally prioritized with beta annealing;
- Double DQN bootstrapping through a target network;
- soft or hard target-network updates;
- epsilon-greedy exploration controlled by the configured reward top-K;
- Adam optimization with Huber loss and optional gradient clipping;
- TD-error clipping and discounted future reward;
- configurable dropout, batch normalization, and L2 regularization.
Candidate-state value learning
For every reachable rotation and column, BrainTris simulates the resulting board, removes
duplicate encoded outcomes, and asks the network for one scalar Q-value per candidate
afterstate. The chosen candidate is executed using the same pre-kick action that produced the
simulation.
A non-terminal Bellman target combines the immediate reward with discounted value from the next
legal candidate set. Terminal experiences contain no future bootstrap value.
Exploration and replay
During training, epsilon controls whether the network exploits its best Q-value or explores.
With K = 0, exploration samples uniformly from all generated legal candidates. With positive K,
it samples uniformly from the K candidates with the highest immediate rewards (or all candidates if fewer exist).
Ties at the cutoff are randomly selected to keep exactly K candidates. Saved runs retain their K value.
Epsilon remains at its starting value during the
warm-up period and then decays toward its configured minimum. Play AI forces epsilon to zero.
Replay updates draw earlier experiences from a fixed-size memory. When PER is enabled, sampling
favors larger TD errors and uses importance weights whose beta value anneals toward 1.0.
Double DQN and target updates
With Double DQN enabled, the online network selects the best next candidate and the target
network evaluates it. The target network can follow the online model through soft Polyak updates
or periodic hard copies, according to the active options.
Optimization and safeguards
Huber loss reduces sensitivity to large value errors, and TD-error limits bound the errors used
during replay. Gradient clipping applies only when a positive clipping threshold is configured;
both norm and value clipping are disabled in the current defaults. Dropout, batch normalization,
and L2 regularization are also disabled by default. These architecture settings apply only when
enabled for the model. The Network strip shows the model
actually in use rather than relying on a fixed Help-page topology description.
Reading an experiment
Game score and learning reward are different quantities. For a non-terminal move, the line
reward is Completed Lines × (cleared lines)Lines Cleared Exponent, plus the enabled
board and penalty-row contributions. With the current default line weight of 0.125 and exponent 2,
one, two, three, and four lines contribute 0.125, 0.5, 1.125, and 2 respectively. A terminal move
receives Game End Penalty instead of those contributions. Zero-weight terms have no effect.
A lower loss or higher Q-value alone does not prove better play. Compare completed-game rows,
survival, and Tetris percentage across many games, and watch Play AI to inspect behavior without
random exploration. Use matching piece seeds and gameplay settings when comparing experiments;
training curves still include exploratory moves, and one successful game is not a reliable trend.
Lab
Well, here we are at the end of an exciting journey, where the simple yet profound world of
Tetris met the complex challenges of artificial intelligence. With the BrainTris application, my
intention was not just to create a game, but also a laboratory where we can explore the power of
Q-Learning and the learning capabilities of neural networks in practice.
I hope that the tools and detailed documentation I have provided will inspire you to experiment.
Feel free to modify the reward system parameters, observe how different parameters influence the
network's behavior, and discover what strategies the AI adopts to master Tetris.
I wish you much success in your experiments and in creating a neural network that plays as
perfectly as possible! I trust that this application will not only provide entertainment but
also a valuable learning opportunity in the fascinating field of artificial intelligence and
machine learning. Explore the possibilities, and let BrainTris help you understand how an AI
thinks and learns!
BrainTris is based on my own original idea and my earlier Java code for Tetris and an AI that
learns to play it. This version was developed from that work with AI assistance from ChatGPT.
Thank you for trying to create a smart artificial intelligence:
β β βββ βββ ββ βββ βββ βββ βββ βββ βββ
βββ βββ βββ βββ βββ ββ βββ β βββ βββ
