_______ __ ________ __
| \ | \ | \ | \
| βββββββ\ ______ ______ \ββ_______ \ββββββββ ______ \ββ _______
| ββ__/ ββ/ \ | \| \ \ | ββ / \| \/ \
| ββ ββ ββββββ\ \ββββββ\ ββ βββββββ\ | ββ | ββββββ\ ββ βββββββ
| βββββββ\ ββ \ββ/ ββ ββ ββ | ββ | ββ | ββ \ββ ββ\ββ \
| ββ__/ ββ ββ | βββββββ ββ ββ | ββ | ββ | ββ | ββ_\ββββββ\
| ββ ββ ββ \ββ ββ ββ ββ | ββ | ββ | ββ | ββ ββ
\βββββββ \ββ \βββββββ\ββ\ββ \ββ \ββ \ββ \ββ\βββββββ
Welcome to the BrainTris Lab!
BrainTris is an in-browser Tetris and reinforcement-learning laboratory. You can play manually,
inspect the configured reward heuristic, train a Deep Q-Network, or watch a trained network
play. The project is designed for experimenting with
Q-Learning
and observing how reward design and training settings change behavior.
Pre-trained network
The download above points to the root brain.bbt model. Play AI
first tries the model saved in this browser. If no browser model exists, it automatically tries
the same /brain.bbt file used by this download link.
What the network sees
Each candidate placement is encoded as four 22 Γ 10 visual channels:
- the fixed stack after the candidate move;
- the tetromino placed by that move;
- column-shadow holes below occupied cells;
- the next tetromino, encoded by shape or piece ID according to the selected option.
The network receives visual grid data rather than handcrafted board metrics. Board metrics are
used by the reward function and for diagnostics, but are not appended to the neural-network
input.
Tetris
Complete horizontal lines to remove them and keep the stack below the top of the board. Clearing
four lines at once is counted as a Tetris. Optional penalty rows and extra tetrominoes can make
the environment harder during experiments.
Game and Learning Modes
The application offers four modes:
- Human: manual Tetris;
- Play Heuristic: the configured reward function chooses moves;
- Train: collects experience and updates the neural network;
- Play AI: a trained network selects moves without exploration.
Human play
Play with the keyboard. The Human and AI play surfaces are rendered by PixiJS.
- ← / →: move;
- ↓: soft drop or accelerate AI playback;
- ↑ or SPACE: rotate;
- P: pause or resume;
- M: toggle music.
Lines and placed pieces contribute to the score. The level increases as lines are cleared, and
the game ends when the next tetromino cannot enter the board.
Play Heuristic
Plays without loading or querying a neural network. It evaluates legal placements with the
currently configured reward function and a short look-ahead, making it useful for inspecting
reward behavior before or during training experiments.
Network training (Train mode)
Training enumerates legal placements, stores encoded experiences in replay memory, and performs
periodic DQN updates. Training intentionally runs without rendering the game board; the DOM
training panel and charts provide feedback instead.
The Train submenu contains:
-
Continue: resume the saved training when one exists;
-
Start New: if saved training exists, confirm its deletion,
then open Training Options; without saved training it opens the options directly. The new
training starts only when Apply is selected;
-
Save: export either a complete training archive or an
inference-only network;
-
Save & Stop: finish the same export first, then run the
normal Stop lifecycle without resuming training between the two operations;
-
Load: restore a .bbt file
selected from disk or dropped into the load area;
- Reset: reset the current training state after confirmation.
Save files
A complete training archive contains model weights, training counters and options, charts, and
replay-buffer data. Its filename starts with the training ID and also includes a compact
six-digit save time and the rounded moving-average rows.
Inference-only export contains just the model and also starts with the training ID. Rename it
to brain.bbt when placing it in the site root for Play AI.
Play AI
Play AI first loads the network cached in this browser. If none is available, it fetches
/brain.bbt from the site root. It then disables exploration and
lets the network choose the highest-valued legal placement. Holding the down arrow accelerates
playback and suppresses nonessential effects.
Stop
Stops the active mode and returns to idle. Stopping training saves its browser state and model;
file export remains a separate action.
Other menu actions
-
Options edits reward, learning, gameplay, rendering, and
monitoring settings.
-
Delete all data removes locally saved training state,
charts, and browser model data, then reloads the application. It does not delete the site's
root brain.bbt.
- The theme selector changes the application color theme.
Training Options
Options edits the configuration used by the next training run
or updates the active training configuration. The question-mark buttons beside individual
fields provide short field-level explanations.
Reward and Punishment
Positive values reward a result and negative values penalize it. The active reward inputs are:
- Completed Lines: reward weight for clearing lines;
-
Lines Cleared Exponent: makes multi-line clears scale
nonlinearly;
-
Average Height: reward or penalty based on the stack's
average column height;
-
Banded Average Height: the lower 6 average-height units use
10% of the configured weight, the middle 7 use 50%, and the upper 7 use the full configured weight.
Every band scales continuously with height and accumulates into the total reward;
- Holes: penalty for covered empty cells;
- Bumpiness: penalty for differences between adjacent column heights;
-
Penalty Row: event reward or penalty when a penalty row is
inserted;
- Game End Penalty: terminal penalty when the game ends.
Learning
-
Target Update selects hard or soft synchronization.
-
Target Update Frequency controls periodic hard copies.
-
Target Update Tau controls soft Polyak updates.
-
Next Piece Channel Mode selects shape-at-spawn or board-wide
piece-ID encoding.
-
Use PER (Prioritized Replay) enables prioritized sampling
and importance-sampling weights.
- Batch Size (max 512) controls samples per replay update.
-
Replay Frequency (steps) controls environment steps between
training updates.
Statistical Moving Average Windows
The short and long windows drive Average Rows smoothing and the displayed learning trend. They
are observational statistics, not additional neural-network inputs.
- Moving Avg. Window: short-term window;
- Long-Term Window: comparison window for the trend.
Gameplay, Rendering, and Monitoring
-
Penalty Rows: enables periodic bottom-row insertion;
-
Extra Tetrominoes: adds non-classic pieces;
- Block Render Style: changes PixiJS block appearance;
-
Stability Panel: shows the optional extended replay and
training diagnostics.
Operations
- Reset to Defaults: restore default option values;
- Apply: validate and apply the displayed values;
- Cancel: close without applying the displayed changes.
Training Panel and Statistics
Training uses a monitoring panel instead of rendering the game board. Updates are intentionally
sampled at different cadences so charts and diagnostics do not dominate training time.
Network topology
Shows the active input shape, convolutional branches, compression stage, flattened feature
count, and dense layers.
AI Vision
Periodically displays the encoded candidate board currently supplied to the network.
Statistics
- Average Rows and trend compare the configured moving windows;
- Games, current lines, total pieces, and total lines show workload;
- Tetrises shows count and percentage of four-line clears;
- Max Lines and Max Level show records;
-
Env. upd. / s and
Train / s separate environment throughput from replay
updates.
Training Data
Shows the latest batch's terminal ratio, reward mean, target mean, TD mean, predicted-next mean,
replay-buffer size, beta, epsilon, gamma, and learning rate. These are raw diagnostics for the
current replay batch, not a single overall quality score.
Charts
-
Average Rows / Tetris %: smoothed rows and four-line-clear
percentage on independently scaled axes; the Tetris percentage uses a tightly padded range
for its complete visible history without rescaling Average Rows;
- Q-Value: selected action-value trend;
- Loss: replay loss trend;
-
Gradient: gradient norm with the configured clipping
threshold as reference.
Stability panel
When enabled, adds Done Ratio, R Mean, Targets Mean, and TD Mean charts plus measured replay/PER
statistics such as buffer fill, rolling coverage, sample age, effective sample size, priority
concentration, TD clipping, and importance-weight range. It reports measurements rather than an
automatic health grade or tuning recommendation.
Reward options
Shows the effective line, exponent, average-height mode, holes, bumpiness, penalty-row, and
game-end weights used by the active run.
Training Methods Used
The active implementation is a value-based DQN over complete legal placements. Its main
components are:
- a four-channel convolutional encoder and scalar Q-value output;
- encoded experience replay, optionally prioritized with beta annealing;
- Double DQN bootstrapping through a target network;
- soft or hard target-network updates;
- epsilon-greedy, reward-filtered exploration during training;
- Adam optimization with Huber loss and gradient clipping;
- TD-error clipping and discounted future reward;
- configurable dropout, batch normalization, and L2 regularization.
Candidate-state value learning
For every reachable rotation and column, BrainTris simulates the resulting board, removes
duplicate encoded outcomes, and asks the network for one scalar Q-value per candidate
afterstate. The chosen candidate is executed using the same pre-kick action that produced the
simulation.
A non-terminal Bellman target combines the immediate reward with discounted value from the next
legal candidate set. Terminal experiences contain no future bootstrap value.
Exploration and replay
During training, epsilon controls whether the network exploits its best Q-value or explores.
Reward-filtered exploration samples from the strongest immediate-reward candidates rather than
blindly choosing from every legal placement. Epsilon remains at its starting value during the
warm-up period and then decays toward its configured minimum. Play AI forces epsilon to zero.
Replay updates draw earlier experiences from a fixed-size memory. When PER is enabled, sampling
favors larger TD errors and uses importance weights whose beta value anneals toward 1.0.
Double DQN and target updates
With Double DQN enabled, the online network selects the best next candidate and the target
network evaluates it. The target network can follow the online model through soft Polyak updates
or periodic hard copies, according to the active options.
Optimization and safeguards
Huber loss reduces sensitivity to large value errors. TD-error limits and GPU-side gradient
clipping bound extreme updates. Dropout, batch normalization, and L2 are architecture settings:
they apply only when enabled by the current configuration. The Network strip shows the model
actually in use rather than relying on a fixed Help-page topology description.
Lab
Well, here we are at the end of an exciting journey, where the simple yet profound world of
Tetris met the complex challenges of artificial intelligence. With the BrainTris application, my
intention was not just to create a game, but also a laboratory where we can explore the power of
Q-Learning and the learning capabilities of neural networks in practice.
I hope that the tools and detailed documentation I have provided will inspire you to experiment.
Feel free to modify the reward system parameters, observe how different parameters influence the
network's behavior, and discover what strategies the AI adopts to master Tetris.
I wish you much success in your experiments and in creating a neural network that plays as
perfectly as possible! I trust that this application will not only provide entertainment but
also a valuable learning opportunity in the fascinating field of artificial intelligence and
machine learning. Explore the possibilities, and let BrainTris help you understand how an AI
thinks and learns!
Coding was assisted by the Gemini and ChatGPT agents.
Thank you for trying to create a smart artificial intelligence:
β β βββ βββ ββ βββ βββ βββ βββ βββ βββ
βββ βββ βββ βββ βββ ββ βββ β βββ βββ
