I’ve trained a model in python and would like to server it on a high throughput webserver. Is it thread safe?
Would it work in parallel? Or would it lock?
I’ve trained a model in python and would like to server it on a high throughput webserver. Is it thread safe?
Would it work in parallel? Or would it lock?
It is thread safe, but it uses a lock, so your concurrent requests will be serialized.
We recently added capacity to support in-place prediction in #5389 but it is currently only available from Python.