2026-04-15 Xcode Derived Data, TMPDIR & xcresult Isolation on Leased Cloud Mac CI

Mobile release teams leasing Mac mini M4 builders across Hong Kong, Tokyo, Seoul, Singapore, and US East often blame “slow Xcode” when the real issue is filesystem contention: two pipelines write the same DerivedData slice, a UI test job fills /tmp, or an .xcresult bundle balloons past 6 GB and starves concurrent archives. This 2026-04-15 playbook explains who needs per-job isolation, catalogs concrete failure modes, compares directory strategies in a table, gives an eight-step shell runbook with numeric NVMe budgets, and links to SwiftPM cache guidance plus signing runbooks so your next hop stays deterministic.

Who Benefits from Per-Job DerivedData on Cloud Macs

Isolation matters when three or more concurrent xcodebuild processes can touch the same workspace family in a rolling 24-hour window—typical for trunk-based iOS shops or white-label factories. If every job inherits the default ~/Library/Developer/Xcode/DerivedData, you inherit stochastic SwiftEmitModule failures, stale module maps, and flaky XCTest attachments. Bare-metal Apple Silicon removes CPU noise, but NVMe still has finite write throughput; pushing scratch IO into predictable per-job trees makes capacity planning honest when you compare regional nodes.

Failure Modes We See on Headless SSH Builders

• Index while you build — background indexing from Job A mutates indexes that Job B reads mid-compile, surfacing as “cannot find type in scope” until a clean.
• Shared TMPDIR — SwiftPM and clang leave hundreds of thousands of tiny files; another job’s cleanup script deletes a prefix still referenced by a sibling process.
• xcresult without quotas — UI tests record video + screenshots; three suites can exceed 8 GB per bundle if attachments are not throttled.
• Archive + test interleave — nightlies that archive then immediately run integration tests on the same DerivedData path amplify corruption risk when the archive step aborts halfway.

Isolation Strategies Compared

Approach	Pros	Cons	When to use
Per-job DERIVED_DATA_PATH under /Volumes/builds	Deterministic cleans; easy quotas per pipeline	First build slower without warm cache	Parallel CI on shared Mac mini M4
Read-only warm cache + overlay copy	Faster cold compiles	Complex rsync; harder on SSH-only hosts	Release trains with identical Xcode minors
Separate macOS user accounts	Hard isolation of signing identities	Higher ops cost; more licenses	Regulated tenants mixing apps

NVMe Budget: Numbers You Can Paste into Runbooks

Artifact	Typical steady state	Spike during UI tests
DerivedData (medium app, Debug)	6–14 GB	+ 3 GB scratch
SwiftPM checkout + build	1–4 GB	+ 2 GB if resolving new tags
xcresult bundle	400–900 MB unit tests	3–10 GB with video

Eight-Step Runbook for Leased Cloud Macs

1. Stamp the job — export JOB_ID=$(date +%s)-$RANDOM and ROOT=/Volumes/builds/$JOB_ID; mkdir -p "$ROOT/DerivedData" "$ROOT/tmp" "$ROOT/results".
2. Pin environment — export DERIVED_DATA_PATH="$ROOT/DerivedData", export TMPDIR="$ROOT/tmp", export TEMP="$TMPDIR", export TMP="$TMPDIR".
3. Trap cleanup — register trap 'rm -rf "$ROOT"' EXIT only after verifying no other job reuses the same JOB_ID; for long-lived agents, archive results before delete.
4. xcodebuild — add -resultBundlePath "$ROOT/results/Test-$JOB_ID.xcresult" for tests; for archives, still export IPA to object storage per IPA export guidance.
5. SwiftPM — pass -clonedSourcePackagesDirPath "$ROOT/spm" and align with registry auth from the SwiftPM article above.
6. Parallelism cap — on 24 GB RAM hosts, keep ≤2 heavy UI test jobs concurrent; add a mutex in your queue if scripts cannot enforce it.
7. Upload — ditto -c -k --sequesterRsrc --keepParent "$ROOT/results/Test-$JOB_ID.xcresult" "$ROOT/results/Test-$JOB_ID.zip" then push to your bucket; verify checksum before deleting local zip.
8. Telemetry — log df -h before/after; alert if job wall time > 42 minutes with > 90% disk utilization—usually indicates runaway attachments.

export JOB_ROOT=/Volumes/builds/ci-$CI_PIPELINE_ID mkdir -p "$JOB_ROOT"/{dd,tmp,res,spm} export DERIVED_DATA_PATH="$JOB_ROOT/dd" TMPDIR="$JOB_ROOT/tmp" TEMP="$JOB_ROOT/tmp" TMP="$JOB_ROOT/tmp"

Metrics Hooks Your Platform Team Can Add in One Afternoon

Disk isolation is meaningless if nobody watches it. Emit three gauges from every job wrapper: (1) du -sk "$ROOT" after compile, (2) wall-clock seconds spent inside xcodebuild archive, and (3) free gigabytes on the volume hosting /Volumes/builds. Push them to whatever you already use—Prometheus, CloudWatch, or a simple syslog line ingested by Vector. When median DerivedData size drifts upward by more than 18% week over week, that is usually a signal that someone disabled incremental compilation or added a heavy code generation step, not that you need new hardware yet.

Also log the Xcode build number (xcodebuild -version) alongside JOB_ID. Teams that rotate between 16.2 and 16.3 on the same host have seen incompatible index shards; correlating spikes with upgrades prevents false blame on “the cloud provider.” Finally, if you run nightly xcrun simctl delete unavailable, schedule it outside peak compile windows so it does not contend with the same NVMe queue your jobs use.

Bridging to SwiftPM & Signing Discipline

DerivedData isolation does not replace provisioning hygiene. Teams that rotate certificates weekly should still follow automatic vs manual signing guidance so codesign never pauses for UI prompts. Likewise, if you vendor binaries through SwiftPM, keep Package.resolved under version control and mirror caches per job—mixing a global SourcePackages directory with per-job DerivedData still races on checkout locks.

FAQ: DerivedData on Rented Apple Silicon

Question	Practical answer
Should I symlink DerivedData to NFS?	Avoid unless latency < 2 ms RTT and you accept slower indexing; prefer local NVMe plus artifact upload.
Does Xcode Server legacy layout matter?	Modern CI should ignore /Library/Developer/XcodeServer; always set explicit paths in scripts.
What about Rosetta?	Keep x86_64 simulators off mixed prefixes; isolate per architecture to prevent fat-binary rebuild storms.

Why Regional Mac mini M4 Nodes Still Matter

Isolated directories remove software races, but physics remains: engineers in Tokyo pulling artifacts from US East pay RTT that shows up as minutes of download. Colocate builders near your SCM and testers; MacXCode’s footprint across HK / JP / KR / SG / US exists so you can place /Volumes/builds on the same continent as your Git LFS and signing audit logs. Read help for SSH baselines, then scale concurrency only after disk telemetry is green for 14 consecutive nightly windows.

Bottom line: per-job DERIVED_DATA_PATH + TMPDIR + explicit xcresult paths turn “random red builds” into measurable IO budgets—then you can trust pricing decisions instead of guessing how many Macs you rent next quarter.

Next read (2026-04-25): code coverage, merging xcresult bundles, JUnit export, and PR merge gates on the same isolated paths—so every shard reports comparable coverage, not just a green exit code.

Next read (2026-04-27): multiple Xcode.app builds on one host—DEVELOPER_DIR per job, runner labels, and SDK policy without xcode-select races—pair isolation paths with a pinned active toolchain for every matrix row.