Preheat the oven to 350°F (175°C).
Grease and line an 8-inch round cake pan.
Whisk together the flour, baking powder, cinnamon, nutmeg, and salt.
Beat the butter and sugar until light and fluffy.
Add the eggs one at a time, mixing well after each addition.
Stir in the vanilla extract.
Alternate adding the dry ingredients and milk until a smooth batter forms.
Spread half of the batter into the prepared pan.
Arrange half of the apple slices over the batter.
Add the remaining batter and smooth the top.
Arrange the remaining apple slices decoratively.
Sprinkle with brown sugar and chopped almonds if using.
Bake for 40–45 minutes or until a toothpick comes out clean.
Cool the cake in the pan for 10 minutes.
Transfer to a wire rack to cool slightly.
Warm the caramel sauce slightly.
Drizzle the caramel generously over the cake.
Dust with powdered sugar if desired.
Slice and serve warm or at room temperature.