PF Simulation r34: personal storage with BYOK

This release extends the accepted r34 Python-overlay architecture. The assistant can still inspect the shared Python solver, propose a private correction, show its diff, validate the baseline and candidate numerically, activate a layer stack, run with it, and remove layers. BYOS changes where bulk evidence is retained. It does not restrict those research tools or change the scientific equations. Standalone operation remains available.

What is stored where

BYOS supports AWS S3, operator-approved S3-compatible HTTPS endpoints, and an outbound connector for a user's NAS or local folder. Object names are opaque and tenant-bound. SQL records retain expected lengths and SHA-256 hashes. PF verifies uploaded bytes by reading them back before publication, and verifies downloaded bytes before use. User storage is not a source of executable control commands.

Storage credentials and AI API keys are separate. Storage credentials use a dedicated encryption key. BYOK continues to use the selected user's OpenAI, OpenAI-compatible, DeepSeek, Gemini or Anthropic credential. Model charges, simulation compute and storage are separate costs. Organization compute, concurrency and logical data limits still apply; BYOS does not grant unlimited execution or storage quota.

Operator setup

1. Back up the database and current artifact stores, then run the normal explicit SaaS database migration. Schema version 4 adds storage destinations, trusted references, immutable resource locations and connector transfer journals. It does not rewrite scientific artifacts.
2. Set PF_SAAS_BYOS_ENABLED=true. Inject PF_SAAS_BYOS_CREDENTIALS_KEY as the URL-safe base64 encoding of 32 random bytes. It must differ from the session and AI-credential keys. Keep it stable and backed up; changing it without re-encrypting credentials makes existing encrypted credentials unreadable.
3. Set PF_SAAS_BYOS_STATE_DIR to a private durable shared volume, mounted at the same path on API replicas and workers and supporting cross-process advisory file locks. In production set PF_SAAS_BYOS_SHARED_STATE=true. The existing shared assistant-state requirement still applies. Production requires PostgreSQL and isolated execution; the development SQLite connector uses its own persistent control database to avoid blocking account transactions.
4. Optionally set PF_SAAS_BYOS_COMPATIBLE_ENDPOINTS to a JSON object mapping operator-approved IDs to HTTPS S3 origins or base paths. Clients select those IDs; they cannot submit arbitrary network destinations. Public-address validation, pinned connections, verified TLS, no redirects and no ambient proxy credentials apply. Private NAS connectivity uses the outbound connector.
5. API replicas and workers require the same storage configuration and encryption key. The default maximum object size is 1 GiB, connector chunk size 1 MiB, connector operation timeout 60 seconds and transit quota 10 GiB. Tune the documented PF_SAAS_BYOS_* settings for expected bandwidth and account limits. A timeout leaves a visible failed or pending operation; it never creates a successful scientific result.

The operator's normal storage backend remains available for users who keep managed storage. Disabling BYOS does not redirect old private resources to managed copies: resources already pinned to user storage become unavailable until their connection is restored.

User setup and existing data

Open Account → Your storage. Create a destination, test its connection, then select it as the default. S3 connections accept a restricted access key/secret and optional session token; the storage principal needs read, write and delete access only to its configured bucket/prefix. This release does not implement AWS role-assumption onboarding.

The default applies to new resources. Existing runs and private assistant profiles retain their original destination. Use the assistant's explicit profile migration control to upload existing evidence and remove hosted payload copies only after verification. The authenticated completed-job migration endpoint is POST /api/jobs/{job_id}/storage/migrate with {"destination_id":"..."}; it requires the existing project and administrative permissions. Failed migration preserves the source and can be retried. Running jobs cannot migrate. Destination changes do not implicitly move earlier data.

Disconnecting storage or revoking its credentials prevents access to its pinned resources. Reconnect the same destination to recover access. Avoid manually renaming or editing the opaque objects: deletions make evidence unavailable and changed bytes fail integrity checks. Backups and retention policies remain the storage owner's responsibility.

NAS or local-folder connector

Create a connector destination and request a pairing token in Account settings. Copy that token into a private, mode-600 file on the NAS host. The token is displayed once and remains valid until revoked or replaced; it is not a single-use credential. Do not put it in command-line arguments or shell history.

Copy the pf_storage_connector directory to a trusted Python 3 environment on a Linux/POSIX host that can access the chosen folder. It uses Python's standard library and imports no PF scientific code. From its parent directory run:

python -m pf_storage_connector --server https://your-pf-service.example \
  --destination DESTINATION_ID --root /your/private/pf-objects \
  --token-file /your/private/pf-connector-token

Keep this process running under your usual service manager. It only initiates outbound HTTPS connections to the configured PF origin. It needs no inbound port, tunnel, public NAS address or access to the shared solver. The chosen folder must not contain symlink components. Transfers use confined file descriptors, authenticated leases, chunk limits, hash checks and atomic file publication. A replacement connector can claim expired work; stale leases cannot publish data. Windows-native operation is not supported by this POSIX implementation; use a Linux host with the folder mounted.

Outages and recovery

Normal reads fail clearly if a destination is offline or corrupt. An assistant operation whose computation completed but whose upload failed retains a protected recovery spool and a completion receipt. Retry pending storage finishes the transfer without repeating provider calls, tools or numerical execution. New mutations are blocked until this save completes. Saved conversation and layer records show the pending-storage state.

An API process crash during an operation has an uncertain outcome. Automatic replay is blocked because it could duplicate work. An operator must confirm the abandoned process and any execution containers have stopped, reconcile its metering record, and reconcile the interrupted storage receipt under the private owner lease. Preserve the shared state volume during this recovery. Credential rotation, loss of connectivity or a replica restart is not permission to discard the last retained evidence copy.

Use python -m pf_saas assistant-operations and the existing assistant-reconcile --operation ID --confirm-stopped command for metering. List storage receipts and their owner identifiers with python -m pf_saas assistant-storage-operations; use python -m pf_saas reconcile-assistant-storage --owner OWNER --confirm-process-stopped for an interrupted receipt. The latter verifies recorded evidence, retains untracked crash files for administrator review, records the uncertain outcome and never replays the interrupted provider/tool work. These are operator commands, not browser endpoints.

Validation and deployment scope

The reproducible release gate is release_tools/validate_r34_byos_release.py. Execute it with the SHA-pinned originally attached r34 ZIP using execute --baseline /path/to/original.zip; use verify to verify retained evidence and current source hashes. It runs the previous regression and UI suites, ordinary run/export workflows, storage failure and isolation tests, and five native AI-provider protocols across S3 and NAS lifecycles. The numerical correction is an explicitly authored fault in an isolated fixture, not a defect introduced into the production solver.

Protocol fixtures exercise actual HTTP transfers and the shipped connector, and numerical tests execute real scientific subprocesses. These results do not constitute a live paid-provider, public cloud, production PostgreSQL, Docker or deployed multi-replica load test. Run a deployment smoke test with your own endpoint, credentials, network, shared volumes and isolated execution configuration before directing production traffic to it.