This is starting to sound like a list of things on my allotment, for exactly the same reasons you want to grow them:
Jerusalem artichokes are good - they look like sunflowers, make a good windbreak, but can get slightly invasive if just left (although the bunnies should sort that out). About 6' tall canes / stems in Summer, but they die down completely in Winter.
Blackcurrants and all crosses with gooseberries (Josta, Worcester, etc). The crosses are more like blackcurrant bushes but with bigger, more oval fruit. Makes lovely jam.
White currant - same as redcurrant, but with white fruits. Tends to be a bit less vigorous.
Apples and crab apples. Very pretty flowers. Crab apples look really good in different colour fruits in Autumn.
Roses - climbing, hedging, etc, All parts bunny edible, hips can be made into syrups, etc.
Rosemary, lavender - make nice low hedges
Strawberries - I plant them under everything on the allotment to keep the weeds out, even if they don't fruit under the blackcurrants.
Mint is also useful as ground cover between fruit bushes. Lots of varieties available.
Lilac and buddlia are bunny friendly and can be part of an informal hedge.