>>1600321pic related>wool>fleece
wool is heavy, very long to dry. Fleece is lighter and faster to dry , but I see wool as superior because it's way better in terms of wind protection. The weight is not a factor for everyone.
Light jackets like pic related are warm, light, way less bulky in a bag, wind-tight and, if you throw an extra buck, can be water resistant too, or buy them cheaper as they can be worn under a rain coat. Thsi configuration has the added benefit of being very versatile
warm+rain or wind= t-shirt+ light raincoat
cold and dry= tshirt +light jacket
cold and wet/windy= tshirt+light jacket+ light raincoat (extra layer compared to one heavy warm raincoat)
last set up will have you covered well into freezing temp, you can throw in a thermal long sleeved t-shirt if you're sensitive to cold or if it gets real frisky. Only if you're doing winter camping in below freezing temp will you need an extra fleece for when you stop, and even then, an extra light jacket will work better and be more convenient