Quoted By:
Philosophically, having a machine tell you when you are acting too much like a machine is the single most constructive application of AI
Human brains actually have a self-determined level of consciousness, set by how often you are able to collapse probability states and to what degree of finesse, including those of your own thoughts. You need to consciously maintain your free will. If you do not, external measurements of probabilities take precedent and you are victimized by determinism - an NPC. In the future, if your activity patterns start aligning with what an AI (lacking self measurement routines) would do, you could be warned that you are failing to maintain your consciousness and will stop being a free thinker if you do not start working towards greater self awareness.
Rest assured AI will probably never be able to set its own states as a conscious observer. Souls did not evolve in reality. Rather, souls arranged reality so that they would be able to inhabit it, and AI will lack the structures needed for a soul to set states in it until someone tries running it on a sophisticated quantum computer. If souls begin inhabiting AI based on a quantum computer or an organic brain (a weak form of a quantum computer) we're going to see some shit.
