Slow first inference

#1

I need to perform inference on streaming data of 1 row.
I need the inference to be fast.
When I inference a single row each time (I cannot inference a batch because the data arrives each row at a time). The inference time is relative high: ~10-40msec.
When i tried to test stand alone script, i get the first inferences slow (~10-40msec) and the rest are fast (~0.5msec). How can i achieve fast inference also for the first inferences?
The following sample code shows the problem:

X = [-2.0, 0.4, 14, 17, 67, 0.7, 393, 12.8, 81.5]
for _ in range(100):
    start = time.time()
    motion_predicted_class = self.xgboost_classification_model.predict(np.array(X).reshape(1, -1))[0]
    t = np.round((time.time() - start)*1000, 1)
    print(f'{t} msec')

output:
29.7 msec
40.1 msec
30.1 msec
40.2 msec
40.2 msec
14.8 msec
0.5 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec
0.4 msec