The service account authenticator isn't the only authenticator that should respect API audience. The authentication config structure should reflect that.