Found an interesting paper that may substantially improve
Character.AI: Google researchers are embedding physics simulations onto language models to simulate abstract thinking and generate realistic and cohesive text
https://arxiv.org/pdf/2210.05359.pdf"Successful and effective communication between humans and AI relies on a shared
experience of the world. By training solely on written text, current language models
(LMs) miss the grounded experience of humans in the real-world—their failure
to relate language to the physical world causes knowledge to be misrepresented
and obvious mistakes in their reasoning. We present Mind’s Eye, a paradigm
to ground language model reasoning in the physical world. Given a physical
reasoning question, we use a computational physics engine (DeepMind’s MuJoCo)
to simulate the possible outcomes, and then use the simulation results as part of
the input, which enables language models to perform reasoning"