Cream the brown and white sugars together with the softened butter.
In a medium bowl, combine the flour, baking soda, cinnamon, ginger, nutmeg, and allspice together and gradually add to the butter mixture.
Add the buttermilk and mix well. Gradually add more buttermilk if needed or if the dough seems to dry.
Fold in the chopped pecans.
Bake at 350 degrees for 8-10 minutes.