I’m very sorry about your experience. That’s definitely not what I was aiming for, and I can imagine that was a nasty surprise. Any hang like that is unacceptable, full stop.
My understanding is Metal is currently causing hangs on devices when there is barely enough RAM to fit the model and prompt, but not quite enough to run. Will work on falling back to CPU to avoid this kind of experience much more aggressively than today.
Thank you for taking the time to both try it out and to share your experience; I will use it to ensure it’s better in the future.
Thanks for the response. Unfortunately on my device the behavior makes it impossible to report a bug using a screenshot as requested in the app. I can give you more device info if you want to narrow down the cause.
My understanding is Metal is currently causing hangs on devices when there is barely enough RAM to fit the model and prompt, but not quite enough to run. Will work on falling back to CPU to avoid this kind of experience much more aggressively than today.
Thank you for taking the time to both try it out and to share your experience; I will use it to ensure it’s better in the future.