Federated from workspace · PL-003 · zaixos-ai-platform/docs/GROQ_PRODUCTION_INTEGRATION.md Do not edit canonical truth here — update the source repo, then re-run npm run docs:sync.

---

Groq Production Integration — PL-003 @ 0.8.0

This guide prepares ZAIXOS consumers for production LLM integration using Groq through the ZAIXOS AI Platform (PL-003) only. Products never call provider SDKs directly.

Architecture

Product (Discovery / Proposal)
  → AiGatewayContract
  → AiProviderGatewayAdapter
  → ProviderGateway (retries, fallback, observability)
  → GroqProvider (OpenAI-compatible API)
  → https://api.groq.com/openai/v1

Business truth remains in Catalog / rule engines. The LLM handles conversation language only.

---

Migration checklist

1. Upgrade platform packages

cd zaixos-ai-platform/packages/contracts && composer install
cd ../laravel && composer install
cd ../../..

cd zaixos-company-platform
composer update zaixos/ai-platform zaixos/ai-contracts
php artisan config:clear

Require: zaixos/ai-platform@^0.8.0, zaixos/ai-contracts@^0.2.1

2. Environment variables

Add to .env (never commit secrets):

# Primary provider
AI_DEFAULT_PROVIDER=groq
AI_GROQ_MODE=live
GROQ_API_KEY=your_groq_api_key_here
GROQ_BASE_URL=https://api.groq.com/openai/v1

# Feature flags
AI_CONVERSATION_ENABLED=true
AI_DISCOVERY_ENABLED=true
AI_CONVERSATION_PROVIDER=groq
AI_CONVERSATION_MODEL=llama-3.3-70b-versatile
AI_DISCOVERY_PROVIDER=groq
AI_DISCOVERY_MODEL=llama-3.3-70b-versatile
DISCOVERY_CONVERSATION_ENABLED=true
DISCOVERY_STREAMING_ENABLED=true

# Resilience
AI_HTTP_TIMEOUT_SECONDS=60
AI_HTTP_CONNECT_TIMEOUT_SECONDS=10
AI_HTTP_RETRY_MAX_ATTEMPTS=3
AI_HTTP_RETRY_DELAY_MS=500
AI_HTTP_RETRY_MULTIPLIER=2
AI_FALLBACK_PROVIDERS=stub

# Streaming
AI_USE_LIVE_SSE=true

# Observability
AI_LOG_REQUESTS=true
AI_LOG_STREAM_DELTAS=false
AI_LOG_CHANNEL=stack
LOG_LEVEL=info

3. Verify health

php artisan ai:health
php artisan ai:health groq

Expected (live + valid key): groq | healthy

Stub mode (no key or AI_GROQ_MODE=stub): groq | stub

4. Local development modes

Mode	Configuration	Use case
Stub	AI_DEFAULT_PROVIDER=stub, AI flags off	CI, offline dev
Groq live	AI_GROQ_MODE=live + GROQ_API_KEY	Local conversation testing
Fallback	AI_DEFAULT_PROVIDER=groq, AI_FALLBACK_PROVIDERS=stub	Resilience testing

---

Platform capabilities (0.8.0)

Capability	Implementation
Provider	GroqProvider — OpenAI-compatible chat + SSE streaming
Streaming	True SSE via ProviderHttpTransport::postSseStream()
Retries	Exponential backoff on rate limit, timeout, 5xx (ProviderRetryExecutor)
Rate limits	HTTP 429 → retryable AiProviderErrorCode::RateLimit
Fallback	AI_FALLBACK_PROVIDERS for complete() and stream()
Timeouts	AI_HTTP_TIMEOUT_SECONDS, AI_HTTP_CONNECT_TIMEOUT_SECONDS
Health	php artisan ai:health → GET /models
Structured output	response_format passthrough + StructuredOutputValidator
Tool calling	tools, tool_choice passthrough; ToolCallDelta stream events
Observability	Structured logs: ai.completion.*, ai.stream.*, ai.fallback.*

---

Testing instructions

Platform unit tests

cd zaixos-ai-platform/packages/laravel
composer test

Consumer tests (stub mode — no API key)

cd zaixos-company-platform
php artisan test --testsuite=Unit --filter=Ai
php artisan test app/Modules/AI/Tests
php artisan test app/Modules/Discovery/Tests/Feature/DiscoveryConversationApiTest.php

Manual Groq smoke test

1. Set .env with live Groq credentials (see section 2).
2. Start app: composer run dev
3. Open /discovery and send an initial intent.
4. Confirm streaming architect response and suggestion chips.
5. Check logs for ai.stream.finished with provider: groq.

Health check in CI

Add to deployment pipeline:

php artisan ai:health groq || exit 1

Skip in CI when using stub:

AI_GROQ_MODE=stub php artisan ai:health groq

---

Registered Groq models

Model key	Capabilities
llama-3.3-70b-versatile	chat, completion, function_calling, streaming
llama-3.1-8b-instant	chat, completion, streaming
mixtral-8x7b-32768	chat, completion, function_calling, streaming

Add models in consumer config/ai.php under models.groq if Groq releases new endpoints.

---

Security

• Store GROQ_API_KEY in environment secrets only (.env, vault, CI secrets).
• Never commit API keys to git.
• Rotate keys if exposed.
• Use AI_LOG_STREAM_DELTAS=false in production to avoid logging user content.

---

Troubleshooting

Symptom	Fix
stub responses despite live config	Set AI_GROQ_MODE=live and non-empty GROQ_API_KEY
Model not supported	Use a registered model key from config/ai.php
429 rate limit	Retries are automatic; reduce concurrency or upgrade Groq tier
Empty streaming	Ensure AI_USE_LIVE_SSE=true and DISCOVERY_STREAMING_ENABLED=true
JSON parse failures	Model must support response_format: json_object; use llama-3.3-70b-versatile
Fallback to stub	Check logs for ai.fallback.triggered; verify primary provider health

---

Rollback

AI_DEFAULT_PROVIDER=stub
AI_GROQ_MODE=stub
AI_CONVERSATION_ENABLED=false
AI_DISCOVERY_ENABLED=false

Discovery continues in rule-based passthrough mode with no API dependency.