Damn, lots of angry fucking nerds in here. Yes OP there are obviously things you have to think about such as what the local laws are, whether you own the land, will you get buttfucked by the local rednecks?
That aside, you asked whether it is a feasible and inexpensive thing to do, and the answer is yes. You can make a small shack out of reclaimed pallet wood which is cheap as dirt (or free if you have the time to find it) and just using a coping saw, hammer, nails and maybe a drill you can get yourself a decent enough shack. This is all provided you know at least basic carpentry though otherwise you'll just be living in a leaky, drafty, damn hole and get rekt (or very ill). Watch how this guy did it
https://www.youtube.com/watch?v=1HA4zY8xCyY, this is obviously a sped up compilation if you are doing this alone don't be surprised if it takes you north of 3 months to get this done.
Couple of really important things. Make sure the floor is raised above the ground otherwise you will be living on damp and eventually rotting wood and say goodbye to any insulating property. Walls need to be made with overlapped planks to stop wind and water. Roof should be either metal or overlapped wood with something completely waterproof like a tarp or a polythene sheet underneath. You WILL need a wood burning stove and a way to get firewood if you don't want to be found dead one day in your little fuck shack, alternatively not ensuring a way for carbon monoxide to escape ensures that your death might happen a lot quicker.
You will need a reliable source of water and a way to purify it but short of being a a fucking master hunter there is zero chance you are going to find all your food in the woods so you'll need some kind of income to buy it.
For what it's worth, I think this is a stupid idea since you seem like you have no idea what you're doing and should have at least have a couple of months of proper camping experience before such a venture but I'm not your mum so go mental.
