Oh I completely get that 0 = 0 folds, but if we are comparing square meters then A1 is 1 square meter.

It depends on your point of view on what you are comparing.

    A0 area = 0.5^0 = 1 m^2
    A1 area = 0.5^1 = 0.5 m^2
    A2 area = 0.5^2 = 0.25 m^2
    A3 area = 0.5^3 = 0.125 m^2

If it was your way

    A1 area = 1 m^2
    A2 area = 0.5 = 0.5 ^ (2-1) m^2 
    A3 area = 0.25 = 0.5 ^ (3-1) m^2

Why shift it one way just to shift it the other way in the formula?

If you want to fit the area into the name you can think of it as

A0 = 1/(2^0) m^2

A1 = 1/(2^1) m^2

A2 = 1/(2^2) m^2

And so on

(EDIT for formatting and also to observe a sibling post to mine is making the same point)