Swamp Industrial

Open source Gato replication project: scaled, designed to prove skill transfer across modalities

Ultimate goal: a pre-trained system that can learn Montezuma's Revenge in <100 hours so we can bonk the weak AGI market.

First goal: Gato replication code using modernized transformer, new tokenization scheme according to Oriol Vinyal's last comments about it.

The time is ripe. The ground is soggy. Join us.

The Swamp Team

January 28, 2025