Preheat the oven to 375°F (190°C). Line two baking sheets with parchment paper.
In a medium bowl, whisk together the all-purpose flour, baking soda, baking powder, and salt. Set aside.
In a large bowl, cream together the softened butter and white sugar until smooth and fluffy using an electric mixer.
Beat in the egg, vanilla extract, and almond extract until well combined.
Gradually add the dry flour mixture to the wet ingredients, alternating with the milk, beginning and ending with the flour mixture. Mix until just combined.
Roll the dough into 1-inch balls and place them 2 inches apart on the prepared baking sheets.
Flatten each ball slightly with the bottom of a glass or your palm.
Sprinkle the tops of the cookies with colored sugar sprinkles and rainbow nonpareils as desired.
Bake in the preheated oven for 8 to 10 minutes, or until the edges are just starting to turn golden.
Remove from the oven and let the cookies cool on the baking sheet for 5 minutes before transferring them to a wire rack to cool completely.
Prepare the royal icing according to the package instructions and use it to decorate the cooled cookies as desired.