분류모델 추론중에 session.run() 에서 다음과 같이 오류가 발생합니다.
npu서버에서 도커기반으로 furiosa-sdk를 설치하고 사용하려는 과정입니다.
2023-11-17T07:19:04.365737Z INFO furiosa_rt_core::driver::event_driven::coord: FuriosaRT (v0.10.2, rev: a45bb1a0b, built at: 2023-10-12T06:41:21Z) bootstrapping ...
2023-11-17T07:19:04.371012Z INFO furiosa_rt_core::driver::event_driven::coord: Found furiosa-compiler (v0.10.1, rev: 8b00177, built at: 2023-10-12T07:44:15Z)
2023-11-17T07:19:04.371020Z INFO furiosa_rt_core::driver::event_driven::coord: Found libhal (type: warboy, v0.11.0, rev: 43c901f built at: 2023-08-08T13:18:00Z)
2023-11-17T07:19:04.371025Z INFO furiosa_rt_core::driver::event_driven::coord: [Runtime-0] detected 1 NPU device(s):
2023-11-17T07:19:04.389993Z INFO furiosa_rt_core::driver::event_driven::coord: - [0] npu:1:0 (warboy-b0, 64dpes, firmware: 1.7.2, e1c4288)
2023-11-17T07:19:04.390228Z INFO furiosa_rt_core::driver::event_driven::coord: [Runtime-0] started
2023-11-17T07:19:04.391240Z INFO furiosa::runtime: Saving the compilation log into /home/furiosa/.local/state/furiosa/logs/compiler-20231117071904-ztwyt2.log
2023-11-17T07:19:04.490142Z INFO furiosa_rt_core::driver::event_driven::coord: [Runtime-0] created Sess-3b02cd5b using npu:1:0
2023-11-17T07:19:04.510249Z INFO furiosa_rt_core::driver::event_driven::coord: [Sess-3b02cd5b] compiling the model (target: warboy-b0, 64dpes, file: model_after.onnx, size: 145.0 MiB)
2023-11-17T07:19:07.490648Z INFO furiosa_rt_core::driver::event_driven::coord: [Sess-3b02cd5b] the model compile is successful (took 2 secs)
2023-11-17T07:19:07.572771Z INFO furiosa_rt_core::driver::event_driven::coord: [Runtime-0] created 1 NPU threads on npu:1:0 (DRAM: 39.5 MiB/16.0 GiB, SRAM: 15.2 MiB/64.0 MiB)
start input
create runner!!!!!!!!!!!!!
test_samples: 1
real_datas: [array([[[[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]]]], dtype=uint8)]
<class 'list'>
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/furiosa/.local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
result = await app( # type: ignore[func-returns-value]
File "/home/furiosa/.local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
return await self.app(scope, receive, send)
File "/home/furiosa/.local/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in __call__
await super().__call__(scope, receive, send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
raise exc
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
await self.app(scope, receive, _send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette_exporter/middleware.py", line 307, in __call__
await self.app(scope, receive, wrapped_send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
raise exc
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
await self.app(scope, receive, sender)
File "/home/furiosa/.local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
raise e
File "/home/furiosa/.local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
await self.app(scope, receive, send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
await route.handle(scope, receive, send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/home/furiosa/.local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
response = await func(request)
File "/home/furiosa/.local/lib/python3.10/site-packages/mlserver/rest/app.py", line 27, in custom_route_handler
return await original_route_handler(request)
File "/home/furiosa/.local/lib/python3.10/site-packages/fastapi/routing.py", line 274, in app
raw_response = await run_endpoint_function(
File "/home/furiosa/.local/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
return await dependant.call(**values)
File "/home/furiosa/.local/lib/python3.10/site-packages/mlserver/rest/endpoints.py", line 59, in infer
inference_response = await self._data_plane.infer(
File "/home/furiosa/.local/lib/python3.10/site-packages/mlserver/handlers/dataplane.py", line 60, in infer
prediction = await model.predict(payload)
File "/home/furiosa/.local/lib/python3.10/site-packages/mlserver/parallel.py", line 117, in _inner
return await pool.predict(payload)
File "/home/furiosa/.local/lib/python3.10/site-packages/mlserver/parallel.py", line 88, in predict
return await loop.run_in_executor(self._executor, _mp_predict, payload)
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
^CINFO: Shutting down
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.