The phrase gets used loosely enough to mean nothing. Used precisely, it describes an app whose interaction model assumes a probabilistic component - which changes how you design screens, handle state, and decide what the user sees while waiting.
Latency becomes a design constraint
A model call is slow and variable in a way an API call usually is not. On mobile that is a design problem before it is an engineering one: what does the screen show for two seconds, is the result streamed as it arrives, and can the user keep working while it resolves? Deciding this late produces spinners in the middle of otherwise good flows.
Design for the wrong answer
A probabilistic feature will sometimes be confidently incorrect. The interface has to make that recoverable: show what the model used, make editing the output cheaper than redoing the task, and never let a model decision silently commit something the user cannot undo.
- Show the inputs behind a generated result
- Make correction faster than starting over
- Confirm before anything irreversible
Decide what runs on the device
On-device inference removes network latency and keeps data local, at the cost of app size, battery and far less capable models. It is a genuine trade-off rather than a default. Sensitive data and simple classification often belong on the device; open-ended generation usually does not.
Test differently
Deterministic assertions do not work when the output varies between runs. Testing shifts toward evaluation sets and properties that must hold - the response is valid JSON, it names only real products, it never returns an empty state - rather than exact-match comparisons that fail on paraphrase.
None of this requires a new architecture. It requires accepting early that one component is probabilistic, and designing the surrounding experience so that its being wrong is an inconvenience rather than a failure.