I would be worried about what other critter would find it's way in, and perhaps in and out and in and out, using the house at it's convenience..mice, rats, hedgie, gopher...you name your local vermin.
I know people who had cat/dog doors who got more than they bargained for in their home!