Toast the almonds at 350°F (175°C) for 8-10 minutes, then let cool. Grind finely if not using pre-made almond flour.
In a large bowl, whisk together the almond flour, all-purpose flour, baking powder, baking soda, and salt.
In another bowl, cream the softened butter and sugar until light and fluffy.
Beat in the eggs one at a time, then add the almond extract and vanilla extract.
Gradually add the dry ingredients to the butter mixture, alternating with the buttermilk, mixing just until combined.
Pour the batter into a greased and floured 9-inch round cake pan (or line with parchment paper).
Bake at 350°F (175°C) for 30-35 minutes, until a toothpick inserted in the center comes out clean.
Let the cake cool in the pan for 10 minutes, then transfer to a wire rack to cool completely.