Preheat oven to 350°F (175°C). Grease and flour two 9-inch round cake pans.
In a bowl, whisk together flour, baking powder, baking soda, salt, and chai spices.
In a separate bowl, cream together butter, brown sugar, and granulated sugar until light and fluffy.
Beat in eggs one at a time, then stir in vanilla extract.
Alternately add the dry ingredients and brewed tea/milk to the creamed mixture, beginning and ending with the dry ingredients. Mix until just combined.
Pour batter evenly into the prepared cake pans.
Bake for 30-35 minutes, or until a toothpick inserted into the center comes out clean.
Let the cakes cool in the pans for 10 minutes before inverting them onto a wire rack to cool completely.
For the frosting, beat together cream cheese and butter until smooth.
Gradually add powdered sugar, beating until smooth. Stir in ground ginger and vanilla extract.
Once the cakes are completely cool, frost the top of one layer and stack the second layer on top. Frost the entire cake.