Deploy Falcon 2 as an API: Serving Other People Safely

The serving stack

An inference engine behind an HTTP API: Text Generation Inference or vLLM for GPU, llama.cpp for CPU and small machines.
A reverse proxy for authentication, rate limiting and request logging.
A queue or concurrency limit, so a traffic spike degrades latency instead of killing the process.

What a public endpoint must have

Authentication, so you know who is spending your GPU time. Per-caller limits, so one client cannot consume the whole batch. Usage accounting, because you will eventually need to explain what happened. And a maximum token limit, because an unbounded request is an unbounded bill.

Licence and disclosure

The TII Falcon License 2.0 is Apache 2.0-based and its acceptable use policy applies to what you serve, not just to how you deploy. If you resell access, keep the licence and attribution with the model, and read the acceptable use policy as a product requirement rather than a legal footnote.

Cost sanity

The moment you serve other people, GPU hours become your unit cost. Measure cost per thousand tokens on your real traffic before you price anything: a single shared GPU looks free until you add concurrency, retries and idle time.

Sources

falcon2.lol is an independent third-party site. It is not affiliated with, endorsed by, sponsored by or operated by the Technology Innovation Institute, and it does not speak for TII. Every number on this page is attributed to the source listed above.