In a medium bowl, whisk together the sifted flour, baking powder, and salt. Set aside.
In a large bowl, beat the softened butter and granulated sugar together on medium speed until light and fluffy, about 3-4 minutes.
Add the egg, vanilla extract, and almond extract to the butter mixture. Beat until combined.
Gradually add the dry ingredients to the wet ingredients, alternating with the milk, beginning and ending with the dry ingredients. Mix until just combined.
Divide the dough into two equal portions. Flatten each into a disk, wrap in plastic wrap, and refrigerate for at least 1 hour.
Preheat the oven to 350°F (175°C). Line baking sheets with parchment paper.
Roll out one disk of dough on a lightly floured surface to about 1/4 inch thickness.
Using a cookie house template, cut out the house pieces: two walls with door and window cutouts, two roof pieces, and a base if desired.
Transfer the cut pieces to the prepared baking sheets.
Bake for 10-12 minutes or until the edges are lightly golden. Remove from oven and let cool completely on wire racks.
Repeat rolling and cutting with the second disk of dough if needed for additional pieces.
Prepare the royal icing by beating the egg whites and cream of tartar until foamy. Gradually add powdered sugar and beat until stiff peaks form and the icing is smooth.
Divide the royal icing into bowls if using food coloring. Tint as desired.
Use the royal icing as glue to assemble the cookie house: pipe icing along the edges of the walls and press together. Hold until set, then attach the roof pieces similarly.
Decorate the assembled house with remaining royal icing and candy decorations as desired.
Allow the assembled and decorated house to dry completely, at least 4 hours or overnight, before moving.