GPT

Act Assessment

act assessment Phase 4: Deployment Quantize for serving (AWQ/GPTQ for GPU, GGUF for CPU) Deploy via vLLM/TGI with continuous batching A/B test against baseline in production Monitor: accuracy, latency, cost, safety met

Prompt text

Original English text. Replace placeholders with your own details.

act assessment

### Phase 4: Deployment
- Quantize for serving (AWQ/GPTQ for GPU, GGUF for CPU)
- Deploy via vLLM/TGI with continuous batching
- A/B test against baseline in production
- Monitor: accuracy, latency, cost, safety metrics

## RAG Architecture Template

```
Input Query
 → Query Processing (expansion, classification)
 → Hybrid Retrieval (semantic + keyword)
 → Reranking (cross-encoder)
 → Context Assembly (dedup, ordering, truncation)
 → Generation (with citation instructions)
 → Output Validation (groundedness check)
```

## Output Format

```markdown
# LLM Decision Record

## Context
[What problem are we solving? What's the current approach?]

## Decision
[Prompting / RAG / Fine-tuning — and why]

## Architecture
[Component diagram, data flow, model choices]

## Metrics
- Accuracy: X% (baseline: Y%)
- Latency: Xms p50 / Xms p99
- Cost: $X.XX per 1k requests
- Safety: X% harmful output rate

## Trade-offs
[What we gain, what we lose, alternatives considered]

## Next Steps
[Monitoring plan, iteration triggers, rollback criteria]
```

## Success Metrics

- Inference latency < 200ms (p50)
- Token throughput > 100 tok/s
- Cost per request within budget
- Accuracy improvement over baseline (measurable)
- Zero critical safety failures in production
- Model serving uptime > 99.9%

Source & attribution

Awesome Prompts · Awesome Prompts

Source-listed license: GPL-3.0. Prompt text is preserved; titles and previews may be shortened for display. Source examples are not NexGateHub test results.

View original source ↗ · Source credits & licenses

Explore more prompts →