tmnetworks is an independent research and software lab working on the fundamentals of neural networks — activation functions and optimization — and turning applied work into practical, everyday apps. Two of them are already on the Microsoft Store.
Peer-shared research on neural network internals — from how single neurons activate to how whole networks are optimized.
A twist on Leaky ReLU that takes the absolute value of its small negative-slope term, keeping gradients alive for negative inputs and tackling the notorious “dying ReLU” problem. Demonstrated accuracy gains over ReLU and Leaky ReLU on image classification (including COVID-19 chest X-rays), text and tabular benchmarks.
Two classic activations fused into one: ReLU behaviour on positive inputs and Sigmoid on negatives. The combination pairs ReLU’s resistance to vanishing gradients with a smooth, non-zero response below zero — reducing dying neurons while keeping training stable.
A hybrid of Tanh and Leaky ReLU that is differentiable at every point, zero-centred, and guarantees a non-zero gradient for negative inputs. Evaluated against ReLU, LReLU and ALReLU across five diverse datasets, showing consistent gains in accuracy and convergence when learning complex non-linear relationships.
Why commit to one optimizer? OptiRoulette samples update rules from an active pool (AdamW, SGD, Nadam…) as training progresses — with warm-up locking, compatibility-aware learning-rate scaling and failure-aware pool replacement. Up to 5.3× faster convergence and up to ~9.7 points higher test accuracy than an AdamW baseline; pip-installable and drop-in compatible with torch.optim.
Muon gives every singular direction of an update equal trust; EchoMuon prices that trust by each direction’s echo — its support in a second, slower momentum buffer. The result is as much about measurement as about the gate: because gating only ever contracts the step, it moves the method along the learning-rate axis, so a shared grid quietly rescues one arm and not the other. Re-run under corrected selection, most of the original margins fall by 42–80%; what survives is directional rather than a step-size effect. 1,815 runs released.
Applied work, actually shipped. Two apps are live on the Microsoft Store; mobile releases are in progress.
See what your PC really costs you in electricity: live wall power, a per-component breakdown, your own tariff with night rates and tiers, and a permanent local history with forecasts. No account, no cloud, no telemetry — and every Pro feature is free for early adopters. Available in 14 languages.
A media player that still knows what a DVD is: VIDEO_TS folders and ISO images play with their menus and chapters intact, next to every ordinary video and audio format. Subtitles are generated on your own machine by whisper.cpp, the music in a scene can be identified on the spot, and every show remembers where you left it. No account, no cloud, no telemetry. Available in 14 languages.
This website counts visits with Google Analytics. No cookies are stored unless you accept — see the privacy policy.