If you do decide to bring them in for winter then they need to come in now to adjust to house temperatures. If the heating is off in the room they are in then they should be fine for a couple of hours out in a run during the day. If they are going to be in human temp rooms of 21 deg roughly then again on warm days outside a couple of hours is fine. But on very cold days of between 0-5 deg for example you will need to restrict this to more like half an hour or so. It's the sudden change in temp that buns can't cope with, but if they are just out for an hour's hop around and exercise they will keep themselves warm enough.
My buns are house buns at around 21 deg, maybe a bit less in the room they sleep in. As winter comes on we reduce the time spent outside to just an hour in the middle of winter, usually just coming out with us for a quick bit of fresh air and a hop around. On very cold days such as freezing or snow we keep it down to 20 mins half an hour - same as we would tolerate really.