Re-running the assessment and tracking drift

A posture report is a photograph, not a live view. It records what your Microsoft 365 looked like at one moment. What changed since we wrote the first version of this page is who takes the next photograph: on every paid plan, Qorionix now does, once a week, without you asking. On the Free plan, it is still entirely up to you.

The weekly automatic re-check

This closes a gap the earlier version of this page was honest about: a report that only moved when someone remembered to click a button eventually stopped being a headline number anyone trusted. The weekly check exists specifically so your Score reflects something closer to this week's tenant rather than the week you signed up.

Why a report ages faster than you expect

The assessment reads configuration: role assignments, sign-in rules, application consents, devices, licences, mailbox rules. Configuration in a small business feels stable and is not. In a typical month at forty people, somebody joins, somebody leaves, one person is given an administrator role for an afternoon and keeps it, and a department signs up for a tool that asks to read mail. None of that announces itself, and each can move a finding. The weekly check catches this within a week rather than within however long it takes someone to remember to look; it does not catch it instantly.

The report is also not the part of the platform that watches. Detection and alerting run continuously from your collected logs; the posture assessment — automatic or manual — is still a snapshot taken periodically, not a stream. That is why ZQ-M365-LOG-002 exists, and its own wording says it plainly: we can see your directory but your day-to-day activity is not being collected — connecting it is what turns this report into ongoing protection.

When to run it yourself anyway

The weekly automatic check (paid plans) handles the ambient case — the drift that happens because a week passed. Some situations are worth a manual run rather than waiting for it, and on Free, manual is the only option there is.

There is still a cost side to running it yourself repeatedly: discovery is a full enumeration of your tenant against Microsoft's API, so running it twice in an afternoon achieves nothing except moving you closer to Microsoft's rate limits. One run per organisation at a time is allowed, and a second is refused rather than queued — this applies equally to a manual run that happens to land while the automatic one is already in progress.

Starting a run yourself

1. 1

   Check who is asking

   Starting a run is governed by the same permission as configuring an integration. Reading a report is deliberately open much wider, so the colleague who reads it may not be the one who can start the next one — see Who can see and do what.

2. 2

   Confirm the connection is still healthy

   The commonest reason a run fails is a client secret that expired since the last one — the same failure the automatic weekly check hits, which is why it is skip-safe: an unhealthy connector is quietly retried next week rather than raising an alarm. A manual run is refused up front instead, with we cannot reach your Microsoft 365 tenant yet. Changing or removing the Microsoft 365 connection covers renewing it.

3. 3

   Start it, then stop watching

   Discovery targets about half an hour and the assessment takes seconds; the statuses are the same as the first time and are listed in What happens in the first hour. Expect a shorter done pile — the audit log is already on, so its check finds nothing to do. That is progress, not the platform doing less.

What carries over between runs, and what does not

A new run — automatic or manual — does not edit the old report. It produces a new one. Knowing which side of that line a thing falls on saves a good deal of confusion.

Findings do not carry over

Every finding is generated afresh from the new inventory. Nothing is copied forward, marked resolved or reused, so a finding in your next report — whoever or whatever started it — is a new statement about your tenant today, which is exactly why its absence means something.

Changes you already made do carry over

They were real changes to your Microsoft 365 and they stay made. What does not reappear is the item in the list, because the check that produced it now has nothing to complain about.

A change you undid is not quietly re-made

This is the one worth knowing. A re-run re-asserts the safe platform-side settings in the done pile — but a setting you explicitly took back is left alone, because the record of your reversal is held against your organisation rather than against the run that made it. This is true whether the run that would have re-asserted it was yours or the automatic weekly one.

Your decisions carry over as evidence, not as suppression

Approving, rejecting and undoing are written to the hash-chained audit log with an actor and a timestamp. But a rejection records your judgement; it is not a mute button. If the condition is still true the finding is reported again, because the alternative is a report that agrees with your last decision instead of with your tenant.

Your answers to the two business questions carry over

Asset criticality and your out-of-hours escalation preference are stored against your organisation and outlive any one run, so a returning tenant is not asked the same two questions twice.

The pack version travels with the report

Every run, finding and change set records which check pack produced it — m365-2026.08.1 at the time of writing. Findings are only comparable across runs at equal pack version, so if the versions differ, some of the difference is us rather than you.

How a fixed finding is proved fixed

There is no button that marks a finding resolved, and that is deliberate. A finding disappears from the next report — automatic or manual — because the check ran again over fresh data and found nothing, not because anybody asserted it had been dealt with. That is stronger proof than a tick box, because it is re-derived from your own tenant every time. It has one trap in it, and it is the trap this whole knowledge base keeps returning to.

The converse holds too: undoing a change brings its finding straight back on the next run, whichever kind, because reverting a fix restores the condition that produced it. That is covered in Undoing a change Qorionix made, and it is not a defect — a report that stayed quiet about a control you had just switched off would be worse than useless.

Reading the difference between two runs

Here is the unflattering part. Every run is kept and you can open the report for each one, but the platform does not produce the difference between two of them for you. There is no delta view and no per-check comparison — your Score shows a history of the headline number itself, which is the closest thing to a trend line the product gives you today. Comparing two reports in detail is still a job you do by reading them, it takes about ten minutes, and doing it by identifier rather than by headline is what makes it quick — which is the practical reason identifiers are stable and never reused.

The third row is the one people skip and it is often the most informative. A finding present in three consecutive reports with the same names is a decision you have made without recording it; the same finding naming different people every time is a process problem rather than a configuration one.

Each run's own row also carries headline counters — how many findings, how many changes were proposed, how many were applied without asking, how many you accepted and how many are still waiting — alongside the pack version, the dates, and now which of them ran on their own. Those numbers in date order will show you a direction. They will not tell you which finding moved, which is why the identifier-by-identifier read above is still the work.

What drift is, and why every report after the first is where the value is

Drift is the gap that opens between how your systems are configured and how you last agreed they should be. A first report cannot show it, because there is nothing to compare against — it can only say where you stand. Drift is visible from the second report onwards, and the checks most likely to reveal it are the ones where a small edit undoes a large protection.

• A sign-in rule turned back to report-only. ZQ-M365-CA-005 catches this. A report-only policy evaluates, logs what it would have done, and then permits the sign-in — zero protection, while still appearing in the list of policies.
• A rule excluded into irrelevance. ZQ-M365-CA-007 fires when a policy's exclusion list has grown past what an emergency account and a spare would justify. Exclusions are added one person at a time, for good reasons, by people not tracking the cumulative effect.
• A temporary administrator, or a new application consent. Nothing in Microsoft 365 removes a role because the reason for it ended, and an application grant has no expiry and survives every staff departure.

This is the argument the weekly automatic check now answers for every paid tenant without anyone having to remember it: one report is a diagnosis; a report every week is the beginning of an answer to the question your auditor, your insurer and your own management body will actually ask, which is not were you secure in March but are you keeping it that way. On Free, that argument is still yours to act on.

What to do next

1. If you are on a paid plan and have not checked, confirm the weekly automatic re-assessment is on for you from your Score page — it is on by default, but you can turn it off.
2. If you granted a permission after reading your last report, start a run now rather than waiting for the weekly one. When a check cannot run lists what each gap needs.
3. On Free, put a recurring reminder somewhere you will see it: monthly while you are working the change set, quarterly afterwards. The product will not remind you.
4. Before an audit, run the assessment and then produce the evidence bundle from it, in that order.