This operation takes 2-5 seconds on every build, but doesn't actually need to run most of the time. Now we cache it and see if it needs a rebuild (fast) before actually rebuilding (slow).
