>>47482666 hereI figured it out, I had to uninstall tensorflow, install tensorflow-rocm and lion-pytorch, and add these lines to the script
At the top:
HSA_OVERRIDE_GFX_VERSION=10.3.0
for accelerate:
>remove --xformers>replace --use_8bit_adam with --use_lion_optimizerthis would be some useful info to add to the lora training guide rentry