Skip to main content

A credential rotation failed

The exact sentence the console shows for each rotation failure, the technical detail per platform, and how to clean up when the target changed and the vault did not.

Written for whoever runs IT9 min readUpdated

A credential in the vault shows Failing, and the What is happening column carries a sentence explaining why. This article is organised around the sentences you actually see, then gives the technical detail behind each one per platform. The section on a change that landed on the target while the vault kept the old secret is the important one, and it names the cases that are defects rather than misconfiguration.

Two things to know first. A failed rotation is not retried until the next interval — there is no backoff, so on the default 30-day interval a failure means a month of silence. Once the cause is fixed, press Rotate now. And rotation only runs against a target: a credential not attached to one cannot rotate at all.

The two layers of every rotation message

What you read is a sentence written for a person. Beneath it, under Technical detail, is the raw string the adapter returned. The sentence tells you the category of problem; the detail tells you which stage of which protocol gave up. The classification tests for authentication failures before network failures, because getting that order backwards tells an administrator to open a firewall port that is already open.

The five outcome buckets and the console label each one gets.
LabelMeaning
ConnectedWe reached the target and authenticated as the account.
UnreachableThe network did not deliver us to the service.
Sign-in refusedThe service answered and rejected the credential.
Not supportedNo adapter for this combination. A product gap, not your fault.
ErrorEverything else, including a permission problem after sign-in.
The sentences behind those labels, verbatim
Connected to <target> and signed in as "<account>".
Could not reach <host> on port <port>. The host did not answer, so check the
  address, the port, and any firewall between Qorionix and it.
Could not reach the target. Check that the address is correct and that Qorionix is
  allowed to connect to it.
<Target> answered, but rejected the credential for "<account>". The password or key
  stored here is not the one the target expects.
The target rejected the credential. The password or key stored here is not the one
  it expects.
Qorionix cannot test this kind of credential on <platform> yet. It will be stored,
  but not verified.
Connected to <target> as "<account>", but the operation did not complete. The
  account may not have permission to change its own password.

The State column, and its exact sentences

Every credential carries a rotation state. Three of them look like failures and are not: No target, Off and Not supported all mean rotation has never run, for three different reasons. Paused means somebody switched it off. Only Failing is an attempt that went wrong.

Rotation state, console label, and the verbatim summary sentence
unbound      No target       Not attached to a target yet, so Qorionix does not know
                             what this secret opens or how to rotate it.
                             Blocker: Attach this credential to a target first.
unbound      No target       Not attached to a target. A <n>-day interval is
                             recorded on this credential, but it has never caused
                             a rotation: without a target there is no machine to
                             rotate on.
unsupported  Not supported   (the platform catalogue's own refusal, below)
off          Off             Rotation is off. Qorionix can rotate this on <platform>.
paused       Paused          Rotation is paused. It will not run until it is
                             switched back on.
pending      Scheduled       Scheduled every <n> days; first run <when>.
ok           Rotating        Last rotated <when>; next <when>.
ok           Rotating        ...plus, where unconfirmed: Qorionix changed the
                             password but could not sign in with the new one to
                             confirm it.
failing      Failing         Last attempt <when> failed. <reason>
failing      Failing         The last attempt failed and no reason was recorded.

One further blocker is easy to hit: No account is recorded for this credential, so Qorionix does not know whose password to change. A secret with no account name can be stored and cannot be rotated. Enabling rotation refuses the same two conditions in the same words — this credential is not attached to a target yet, so Qorionix does not know which machine to rotate it on and this credential has no account recorded, so Qorionix does not know whose password to change.

Could not reach the target

Unreachable, by platform
linux_ssh dial: <error>
cisco_ios dial <addr>: <error>
postgres connect as "<admin>": <error>
sqlserver dial <addr>: <error>
windows_local do: <error>
ad do: <error>
paloalto_panos keygen as "<account>": <error>
-- and the pre-flight refusals, which are configuration rather than network:
linux_ssh: host and account are required
cisco_ios: host and account are required
postgres: target host is required
sqlserver: target host is required
ad_service_account: domain_controller or host required

All of these are address, port or firewall problems rather than credential problems. The phrases that produce the classification are the ordinary transport ones — connection refused, no such host, i/o timeout, network is unreachable, no route to host, connection reset by peer, context deadline exceeded, dial tcp. Check the address and port in Targets and devices first; a non-standard port is a common cause.

The target answered and refused the credential

This means the secret in the vault is not the secret the target expects — which, on a credential that has never rotated, almost always means somebody changed the password on the machine without telling the vault. The recognised signatures include unable to authenticate, ssh: handshake failed, no supported methods remain, password authentication failed, login failed for user, invalid credentials, access denied, permission denied, bad credentials, the PostgreSQL codes 28P01 and 28000, and, specifically for Cisco IOS, authentication with the target failed.

Connected, then the change was refused

Post-sign-in failures, by platform
linux_ssh <command>: <error>
linux_ssh session: <error>
postgres ALTER ROLE "<account>": <error>
sqlserver ALTER LOGIN "<account>": <error>
windows_local http <status>: <body>
ad http <status>: <body>
cisco_ios rejected "<command>": <device output>
cisco_ios: enable did not reach privileged EXEC mode; check metadata.enable_secret
PAN-OS returned status "<status>": <detail>
paloalto_panos commit job <id> finished with FAIL: <detail>
paloalto_panos commit job <id> did not finish within <budget>
aws http <status>: <body>
create access key: <error>
graph http <status>: <body>
entra_service_principal: Graph addPassword returned no secretText

These are the cases where the account may not have permission to change its own password is usually literally true. The two database messages mean the rotating login signed in and lacks the right to alter that role or login — CREATEROLE on PostgreSQL, ALTER ANY LOGIN on SQL Server. The Windows and Active Directory forms pass the shim's status and body through unedited; a 403 there is nearly always a missing reset-password or delegation right. On Cisco IOS the device's own output is quoted with secrets stripped, and did not reach privileged EXEC mode means the enable secret is wrong. On AWS, suspect a policy without iam:CreateAccessKey, or a user already holding the maximum two keys; on Entra ID, an identity lacking the Graph permission to add a password credential.

Three refusals that are safety guards rather than faults
refusing to set an empty password
generated password is <n> characters, beyond the 128 this engine will interpolate
generated password contains an unsafe character "<rune>" at position <n>; refusing
  to interpolate it into DDL
sqlserver <host> reports it cannot encrypt the connection; refusing to send a
  credential in the clear (set metadata.allow_unencrypted=true to override)
cisco_ios: metadata.privilege must be numeric, got "<value>"

The target changed and the vault did not

This is the dangerous class. The rotation reached the target, the password changed, and something after that point went wrong. The result is recorded as failed — correctly — but the machine has moved on, so access stays broken until you reconcile the two. There are three shapes, and only the second is a defect.

The change landed and could not be confirmed

Verification failed after a successful change
linux_ssh: the password was changed but the new secret does not authenticate for
  "<account>": <error>
postgres: password changed but the new secret does not authenticate for
  "<account>": <error>
sqlserver: password changed but the new secret does not authenticate for
  "<account>": <error>
cisco_ios: password changed but the new secret does not authenticate for
  "<account>": <error>
paloalto_panos: password committed but the new secret does not authenticate for
  "<account>": <error>

Five adapters prove the new secret by signing in with it, and these are their failure messages. The change is on the target; the attempt is recorded as failed, so the vault still holds the previous value. The cleanup is to reset the account on the target back to a known value, then rotate again — one ALTER ROLE or ALTER LOGIN from an administrative session, passwd on Linux, or the device's own console. Do not simply press Rotate now: the next attempt authenticates with the vault's copy first, and that is the copy the target has stopped accepting.

The new secret exists and Qorionix did not keep it

The three messages where a new secret was created and not stored
aws_iam_key: rotation succeeded but old key cleanup failed: <error>
entra_service_principal: new secret <key-id> created but cleanup incomplete: <detail>
paloalto_panos: phash written to the candidate config for "<account>" but
  skip_commit was set, so the new password is not yet active
AWS IAM access key
A new key was created and the old one could not be deleted — so the old key still works and your access is intact. What you have is an orphaned key nobody holds. Cleanup: in IAM, find the newest access key on that user, confirm it is unused, delete it, fix the permission that blocked the deletion, then rotate again.
Microsoft Entra ID service principal
A new client secret was added and removal of the old ones did not complete, so the vault's copy still works. The message names the new credential's key id. Cleanup: delete the password credential with that key id from the app registration, then rotate again.
Palo Alto firewall with skip_commit set
The hash is in the candidate configuration, uncommitted, and Qorionix did not keep it. This is the worst of the three: whoever commits next locks the account to a secret that exists nowhere. Cleanup: revert the candidate configuration rather than committing it, then rotate again with skip_commit cleared. If a commit has already happened, reset the account from the firewall's console.

The vault could not store the new secret

The platform refuses to record success unless it has stored the new secret first, and downgrades the result when it cannot. Both messages state the consequence outright, and both are ours to fix — but they leave you the same reconciliation job as above: reset the account on the target, then rotate again.

the target's password was changed but Qorionix could not store the new secret: the credential sealer was not installed at start-up. The vault and the target now disagree.
the target's password was changed but Qorionix could not store the new secret (<error>). The vault and the target now disagree; rotate again or reset the account manually.

Combinations Qorionix cannot rotate

Platform-catalogue refusals
"<value>" is not a platform Qorionix manages; supported platforms are
  active_directory, aws, cisco_ios, entra_id, generic, linux, panos, postgresql,
  sqlserver, windows
Qorionix can store credentials for Other (no rotation) but cannot rotate them
  automatically
Qorionix cannot rotate a ssh key credential on Linux server; it can rotate password

Not supported is an honest answer, not a failure. The second refusal is the generic platform, display name Other (no rotation), which exists so a secret with no supported target still gets a vault, an audit trail and a checkout flow. The third names the credential types that are rotatable there, so it doubles as the answer to what should I have chosen. Several adapters also change a secret without proving it afterwards, and the state sentence says so rather than implying a confirmation that never happened.

Failures before we contact anything

Scheduler and deployment refusals
Qorionix has no rotation engine called "<adapter>", so this schedule can never run.
  Recreate it choosing the platform of the target.
Qorionix could not open the stored credential for this schedule, so it did not
  attempt to contact the target.
rotation is not available: the service was started without a rotation store
rotation cannot run in this process: it was started without the credential sealer

The first is a schedule naming an engine that does not exist: delete it and enable rotation again from the credential, where the engine is derived rather than typed in. The other three are deployment faults, caught before anything is touched, and are ours.

What to do, in order

  1. Read the sentence, then the technical detail

    The sentence tells you which of five categories you are in, and therefore who has to act. Unreachable is a network job, Sign-in refused is a stored-secret job, Error is usually a permission on the target, Not supported is ours.

  2. Test the target before touching the credential

    The connection test tells you whether the address, port and stored secret work right now, without changing anything. Until a target has been tested, nobody knows whether the credentials stored against it still open it.

  3. Look for the words changed but

    If the detail contains password was changed but, password changed but, password committed but, created but cleanup incomplete or could not store the new secret, stop and reconcile before retrying. Everything else is safe to retry.

  4. Fix the cause, then press Rotate now

    Nothing retries for you, and a failed schedule waits a full interval — 30 days by default. Rotate now is the whole remedy once the cause is gone.

  5. If it says Not supported, plan differently for that secret

    Store it, record its account and target so the audit trail is complete, and rotate it on a calendar reminder. Worse than automation, much better than a schedule that silently never runs.

For how rotation works when it works, read Credential rotation overview. For the setup that prevents most permission failures above: Rotating database credentials and Rotating Linux and SSH credentials.

Was this article wrong?

If a procedure here does not match what you see, or a limit we described has changed, tell us and we will fix the page. Email us about this article, or see how to get help if you need an answer rather than a correction.

Everything in troubleshooting