Small Transformer Hardware