According to one of the founders, "Login with Amazon" doesn't grant access to a user's Amazon account.[1] If they were using it, there would be no need to ask for users' passwords.

1: https://news.ycombinator.com/item?id=8070310