Even if you're not using a proper PKI, you still have to trust someone at the end of the day to not be lying to you. That's ostensibly the point behind CAs and auditing and trust roots.

LE's probably the best we're going to get for a long time. Mozilla's one of the few groups I'd go so far as to say is completely benevolent, and LE is being operated as a public service.