In 8fa3425, apparently before 1.0, the option to disable password as a "required credential" was introduced. Since this concept of "required credentials" was relatively soon marked deprecated (referenced here), the feature is seemingly impossible to use right now.

Having multiple federated IdPs is a common use case for Keycloak as an identity multiplexer. And the existing workarounds are rather poor and usually consist of setting an impossible to fulfill password policy in addition to cosmetic changes through a theme.