I know about variable and command substitution and escaping, it makes sense that those expand inside " but not ' - but history expansion, something seemingly inherent to the interactive REPL, seems out of place here.

I understand your comment. Intuitively I agree with you but in the context of one of the original devs writing bash it would make no sense to tie the history command to the interactive REPL. Treating everything the same would lead to less bugs in "their" system while leaving a caveat for all the userland script writers.

There is no difference between a script and the REPL.

Besides, you'd expect variables to be expanded inside normal strings -- why special-case the other expansions?

I'd expect them to be handled in the same "phase" as alias expansions. See here for one example of hackery with aliases, that take advantage of their different expansion phase:

http://www.chiark.greenend.org.uk/~sgtatham/aliases.html

Special characters are substituted in double quotes. They're treated as literals in single quotes.

Seems pretty consistent to me.

Aliases are handled differently though, before e.g. file glob expansion. I would have expected history to be handled similarly.