Core Analysis¶

Core question: What concrete changes are needed to migrate OpenAI cloud-based applications to Foundry Local, and what are the migration blockers?

Technical Analysis¶

• Typical migration changes:
  1. Switch endpoint/credentials: Point your OpenAI client to the local service endpoint or adopt the C# local SDK.
  2. Model alias mapping: Ensure local model aliases (e.g., phi-3.5-mini, qwen2.5-0.5b) match application expectations.
  3. Streaming & timeout settings: Local latency and throughput differ from cloud; adjust timeouts, retries, and concurrency.
  4. Response field verification: Validate that OpenAI response fields and metadata match assumptions and update parsing logic.

• Potential compatibility pitfalls:

• Missing advanced/proprietary APIs: Foundry Local supports basic OpenAI calls, but cloud-specific features (training/fine-tuning, certain integrations) may not be supported.
• Performance and concurrency limits: Local resources determine concurrency—reevaluate strategies to avoid OOM or CPU/disk bottlenecks.
• SDK behavior differences: The C# in-process API and HTTP service may differ in error codes, cancellation handling, etc., requiring integration testing.

Practical Recommendations¶

1. Migrate from least-coupled features: Begin with simple chat/completion endpoints and verify outputs.
2. Maintain a fallback path: Keep a cloud toggle during early migration to compare model quality and availability.
3. Create a validation suite: Automate tests for request/response compatibility, latency and memory usage to avoid runtime surprises.

Important Notes¶

Important: Do not assume all OpenAI features are available locally. Enumerate the OpenAI features your app depends on and confirm Foundry Local support before full migration.

Summary: Migration is low-effort for basic chat/completion apps (endpoint switch, model alias checks, timeout tuning). Apps relying on advanced cloud features require deeper compatibility testing and performance validation.