As others have said just take things slowly, if bun is happy to take treats from you then give them more fuss whilst you're giving them treats, and use the treats as rewards.
One of my buns was quite scaredy when we got her, so we started off getting her to sit near us and have treats, then stroked her, then we picked her up and put her straight down again, then gradually picked her up more and for a little longer, our end result was picking her up once a day and checking her eyes/ears/bums. We used treats all through the process and even now (5 yrs later) she always gets a treat whenever we put her back on the floor after picking her up. So now she tolerates it and doesn't get stressed.