If you wrap a rabbit in a towel so they cant kick, you can cut their nails with special small-animal or small-dog nail clippers. I get one person to hold the rabbit on their lap, whilst I kneel in front and clip the nails. If your rabbit has light nails, you can see a vein, and you should never cut too near this vein or it will bleed a lot. If your rabbit has dark nails it is harder to get it right.
Best thing is to get the vet to show you first if you want to do it at home. My vet will do it for an extra £5.
Be careful you don't trance your rabbit whilst doing it (putting them on their back so they go still and 'play dead') as this is very stressful for them and can ruin your relationship with them.
edit: if you suspect your rabbit might have overgrowing teeth, or if her eating habits have changed or she wont eat much hay, I would get her teeth (both front and back) checked out by a vet, as it can become serious.