Full log
201/333 Test #201: interface.MessagesFormatTest.FormatAssistantReasoningContentInjectedToChatML ......................... Passed 0.01 sec
Start 202: interface.MessagesFormatTest.FormatAssistantReasoningOnlyNoContent
202/333 Test #202: interface.MessagesFormatTest.FormatAssistantReasoningOnlyNoContent ................................... Passed 0.01 sec
Start 203: interface.MessagesFormatTest.FormatQwenThinkingDisabledAppendsEmptyBlock
203/333 Test #203: interface.MessagesFormatTest.FormatQwenThinkingDisabledAppendsEmptyBlock ............................. Passed 0.01 sec
Start 204: interface.MessagesFormatTest.FormatQwen3BaseThinkingEnabledNoOpenTag
204/333 Test #204: interface.MessagesFormatTest.FormatQwen3BaseThinkingEnabledNoOpenTag ................................. Passed 0.01 sec
Start 205: interface.MessagesFormatTest.FormatQwen35ThinkingEnabledAppendsOpenTag
205/333 Test #205: interface.MessagesFormatTest.FormatQwen35ThinkingEnabledAppendsOpenTag ............................... Passed 0.01 sec
Start 206: interface.MessagesFormatTest.FormatQwen35ThinkingDisabledAppendsEmptyBlock
206/333 Test #206: interface.MessagesFormatTest.FormatQwen35ThinkingDisabledAppendsEmptyBlock ........................... Passed 0.01 sec
Start 207: interface.MessagesFormatTest.FormatNonQwen3ModelNoThinkingSuffix
207/333 Test #207: interface.MessagesFormatTest.FormatNonQwen3ModelNoThinkingSuffix ..................................... Passed 0.01 sec
Start 208: interface.MessagesFormatTest.FormatQwenEndsWithAssistantNoThinkingSuffix
208/333 Test #208: interface.MessagesFormatTest.FormatQwenEndsWithAssistantNoThinkingSuffix ............................. Passed 0.01 sec
Start 209: interface.MessagesFormatTest.FormatQwenEndsWithAssistantReasoningDoesNotAppendGenerationSuffix
209/333 Test #209: interface.MessagesFormatTest.FormatQwenEndsWithAssistantReasoningDoesNotAppendGenerationSuffix ....... Passed 0.01 sec
Start 210: interface.MessagesFormatTest.ParseThinkingDefaultWhenOmitted
210/333 Test #210: interface.MessagesFormatTest.ParseThinkingDefaultWhenOmitted ......................................... Passed 0.01 sec
Start 211: interface.MessagesFormatTest.ParseThinkingDisabled
211/333 Test #211: interface.MessagesFormatTest.ParseThinkingDisabled ................................................... Passed 0.01 sec
Start 212: interface.MessagesFormatTest.ParseThinkingEnabled
212/333 Test #212: interface.MessagesFormatTest.ParseThinkingEnabled .................................................... Passed 0.01 sec
Start 213: interface.MessagesFormatTest.ParseTopLevelEnableThinkingFalse
213/333 Test #213: interface.MessagesFormatTest.ParseTopLevelEnableThinkingFalse ........................................ Passed 0.01 sec
Start 214: interface.MessagesFormatTest.ParseTopLevelEnableThinkingTrue
214/333 Test #214: interface.MessagesFormatTest.ParseTopLevelEnableThinkingTrue ......................................... Passed 0.01 sec
Start 215: interface.MessagesFormatTest.ParseExtraBodyEnableThinkingAlone
215/333 Test #215: interface.MessagesFormatTest.ParseExtraBodyEnableThinkingAlone ....................................... Passed 0.01 sec
Start 216: interface.MessagesFormatTest.ParseThinkingAllSourcesConsistent
216/333 Test #216: interface.MessagesFormatTest.ParseThinkingAllSourcesConsistent ....................................... Passed 0.01 sec
Start 217: interface.MessagesFormatTest.ParseThinkingAllSourcesConsistentDisabled
217/333 Test #217: interface.MessagesFormatTest.ParseThinkingAllSourcesConsistentDisabled ............................... Passed 0.01 sec
Start 218: interface.MessagesFormatTest.ParseThinkingNonBooleanRejected
218/333 Test #218: interface.MessagesFormatTest.ParseThinkingNonBooleanRejected ......................................... Passed 0.01 sec
Start 219: interface.MessagesFormatTest.ParseThinkingTypeMissingOrNotStringRejected
219/333 Test #219: interface.MessagesFormatTest.ParseThinkingTypeMissingOrNotStringRejected ............................. Passed 0.01 sec
Start 220: interface.MessagesFormatTest.ParseThinkingConflictRejected
220/333 Test #220: interface.MessagesFormatTest.ParseThinkingConflictRejected ........................................... Passed 0.01 sec
Start 221: interface.MessagesFormatTest.ParseReasoningFormatDefaultsToAuto
221/333 Test #221: interface.MessagesFormatTest.ParseReasoningFormatDefaultsToAuto ...................................... Passed 0.01 sec
Start 222: interface.MessagesFormatTest.ParseReasoningFormatExplicitModes
222/333 Test #222: interface.MessagesFormatTest.ParseReasoningFormatExplicitModes ....................................... Passed 0.01 sec
Start 223: interface.MessagesFormatTest.ResolveThinkingStyleFromModelsJson
223/333 Test #223: interface.MessagesFormatTest.ResolveThinkingStyleFromModelsJson ...................................... Passed 0.01 sec
Start 224: interface.MessagesFormatTest.ResolveThinkingStyleQwen3DotXFromJson
224/333 Test #224: interface.MessagesFormatTest.ResolveThinkingStyleQwen3DotXFromJson ................................... Passed 0.01 sec
Start 225: interface.MessagesFormatTest.ResolveThinkingStyleUnregisteredModelReturnsNone
225/333 Test #225: interface.MessagesFormatTest.ResolveThinkingStyleUnregisteredModelReturnsNone ........................ Passed 0.01 sec
Start 226: interface.MessagesFormatTest.ResolveThinkingStyleReturnsNoneForBadConfig
226/333 Test #226: interface.MessagesFormatTest.ResolveThinkingStyleReturnsNoneForBadConfig ............................. Passed 0.06 sec
Start 227: interface.ToolCallRequestTest.ToolsNotArrayRejected
227/333 Test #227: interface.ToolCallRequestTest.ToolsNotArrayRejected .................................................. Passed 0.01 sec
Start 228: interface.ToolCallRequestTest.ToolsObjectRejected
228/333 Test #228: interface.ToolCallRequestTest.ToolsObjectRejected .................................................... Passed 0.01 sec
Start 229: interface.ToolCallRequestTest.ToolChoiceNumberRejected
229/333 Test #229: interface.ToolCallRequestTest.ToolChoiceNumberRejected ............................................... Passed 0.01 sec
Start 230: interface.ToolCallRequestTest.InvalidToolChoiceObjectRejected
230/333 Test #230: interface.ToolCallRequestTest.InvalidToolChoiceObjectRejected ........................................ Passed 0.01 sec
Start 231: interface.ToolCallRequestTest.ToolChoiceArrayRejected
231/333 Test #231: interface.ToolCallRequestTest.ToolChoiceArrayRejected ................................................ Passed 0.01 sec
Start 232: interface.ToolCallForwardTest.ToolsAndToolChoiceForwardsAsToolCallTask
232/333 Test #232: interface.ToolCallForwardTest.ToolsAndToolChoiceForwardsAsToolCallTask ............................... Passed 0.06 sec
Start 233: interface.ToolCallForwardTest.ToolChoiceDefaultsToAutoWhenOmitted
233/333 Test #233: interface.ToolCallForwardTest.ToolChoiceDefaultsToAutoWhenOmitted .................................... Passed 0.06 sec
Start 234: interface.ToolCallForwardTest.ToolChoiceAutoForwardsAsToolCallTask
234/333 Test #234: interface.ToolCallForwardTest.ToolChoiceAutoForwardsAsToolCallTask ................................... Passed 0.06 sec
Start 235: interface.ToolCallForwardTest.ToolChoiceFunctionObjectForwardsAsToolCallTask
235/333 Test #235: interface.ToolCallForwardTest.ToolChoiceFunctionObjectForwardsAsToolCallTask ......................... Passed 0.06 sec
Start 236: interface.ToolCallForwardTest.EmptyToolsArrayDoesNotForwardAsToolCall
236/333 Test #236: interface.ToolCallForwardTest.EmptyToolsArrayDoesNotForwardAsToolCall ................................ Passed 0.06 sec
Start 237: interface.ToolCallForwardTest.MissingToolsDoesNotForwardAsToolCall
237/333 Test #237: interface.ToolCallForwardTest.MissingToolsDoesNotForwardAsToolCall ................................... Passed 0.06 sec
Start 238: interface.ToolCallForwardTest.ToolChoiceWithoutToolsDoesNotForwardAsToolCall
238/333 Test #238: interface.ToolCallForwardTest.ToolChoiceWithoutToolsDoesNotForwardAsToolCall ......................... Passed 0.06 sec
Start 239: api.GewuSmartServeApiTest.NullArgumentsReturnInvalidArgument
239/333 Test #239: api.GewuSmartServeApiTest.NullArgumentsReturnInvalidArgument ......................................... Passed 0.01 sec
Start 240: api.GewuSmartServeApiTest.EmptyArgumentsAreRejectedAtThePublicBoundary
240/333 Test #240: api.GewuSmartServeApiTest.EmptyArgumentsAreRejectedAtThePublicBoundary ............................... Passed 0.01 sec
Start 241: api.GewuSmartServeApiTest.ModelsDirectoryRequiresCanonicalAbsolutePath
241/333 Test #241: api.GewuSmartServeApiTest.ModelsDirectoryRequiresCanonicalAbsolutePath ............................... Passed 0.01 sec
Start 242: api.GewuSmartServeApiTest.ModelsDirectoryRejectsGroupOrWorldWritableRoot
242/333 Test #242: api.GewuSmartServeApiTest.ModelsDirectoryRejectsGroupOrWorldWritableRoot .............................***Skipped 0.01 sec
Start 243: api.GewuSmartServeApiTest.InitializeFinalizeAreIdempotent
243/333 Test #243: api.GewuSmartServeApiTest.InitializeFinalizeAreIdempotent ............................................ Passed 0.59 sec
Start 244: api.GewuSmartServeApiTest.FinalizeOwnsOuterLeaseUntilPublicEntryReturns
244/333 Test #244: api.GewuSmartServeApiTest.FinalizeOwnsOuterLeaseUntilPublicEntryReturns .............................. Passed 0.63 sec
Start 245: api.GewuSmartServeApiTest.FailedFinalizeMustBeRetriedBeforeInitialize
245/333 Test #245: api.GewuSmartServeApiTest.FailedFinalizeMustBeRetriedBeforeInitialize ................................ Passed 0.64 sec
Start 246: api.GewuSmartServeApiTest.FinalizeWaitsForActiveSdkCall
246/333 Test #246: api.GewuSmartServeApiTest.FinalizeWaitsForActiveSdkCall .............................................. Passed 0.63 sec
Start 247: api.GewuSmartServeApiTest.FinalizeIsRejectedFromSdkCallback
247/333 Test #247: api.GewuSmartServeApiTest.FinalizeIsRejectedFromSdkCallback .......................................... Passed 0.64 sec
Start 248: api.GewuSmartServeApiTest.CACertificatePathRoundTrips
248/333 Test #248: api.GewuSmartServeApiTest.CACertificatePathRoundTrips ................................................ Passed 0.04 sec
Start 249: api.GewuSmartServeApiTest.InvalidJsonIsRejectedBeforeServiceInitialization
249/333 Test #249: api.GewuSmartServeApiTest.InvalidJsonIsRejectedBeforeServiceInitialization ........................... Passed 0.02 sec
Start 250: api.GewuSmartServeApiTest.ChatCompletionsStreamInvalidJsonRejected
250/333 Test #250: api.GewuSmartServeApiTest.ChatCompletionsStreamInvalidJsonRejected ................................... Passed 0.01 sec
Start 251: api.GewuSmartServeApiTest.FreeHandlesNullAndValidPointers
251/333 Test #251: api.GewuSmartServeApiTest.FreeHandlesNullAndValidPointers ............................................ Passed 0.60 sec
Start 252: api.GewuSmartServeApiTest.VersionAndListEnginesReturnValidData
252/333 Test #252: api.GewuSmartServeApiTest.VersionAndListEnginesReturnValidData ....................................... Passed 0.64 sec
Start 253: api.GewuSmartServeApiTest.ChatCompletionsRejectsMalformedOrIncompleteRequests
253/333 Test #253: api.GewuSmartServeApiTest.ChatCompletionsRejectsMalformedOrIncompleteRequests ........................ Passed 0.63 sec
Start 254: api.GewuSmartServeApiTest.ConfigJsonValidation
254/333 Test #254: api.GewuSmartServeApiTest.ConfigJsonValidation ....................................................... Passed 0.10 sec
Start 255: api.GewuSmartServeApiTest.ConfigJsonCopiesInputBuffer
255/333 Test #255: api.GewuSmartServeApiTest.ConfigJsonCopiesInputBuffer ................................................ Passed 0.60 sec
Start 256: api.GewuSmartServeApiTest.SetModelsConfigPathLoadsConfigFile
256/333 Test #256: api.GewuSmartServeApiTest.SetModelsConfigPathLoadsConfigFile ......................................... Passed 0.64 sec
Start 257: api.GewuSmartServeApiTest.SetModelsConfigPathRejectsRelativeAndSymlinkPaths
257/333 Test #257: api.GewuSmartServeApiTest.SetModelsConfigPathRejectsRelativeAndSymlinkPaths .......................... Passed 0.04 sec
Start 258: api.GewuSmartServeApiTest.SetModelsConfigJsonRejectsSymlinkedDerivedPathsBeforeCommit
258/333 Test #258: api.GewuSmartServeApiTest.SetModelsConfigJsonRejectsSymlinkedDerivedPathsBeforeCommit ................ Passed 0.61 sec
Start 259: api.GewuSmartServeApiTest.CachedConfigJsonDoesNotReloadRelativePathFromWorkingDirectory
259/333 Test #259: api.GewuSmartServeApiTest.CachedConfigJsonDoesNotReloadRelativePathFromWorkingDirectory .............. Passed 0.64 sec
Start 260: api.GewuSmartServeApiTest.OversizedModelsConfigInputsAreRejectedWithoutReplacingCurrentConfig
260/333 Test #260: api.GewuSmartServeApiTest.OversizedModelsConfigInputsAreRejectedWithoutReplacingCurrentConfig ........ Passed 0.64 sec
Start 261: api.GewuSmartServeApiTest.SetModelsConfigPathRejectsInvalidFilesImmediately
261/333 Test #261: api.GewuSmartServeApiTest.SetModelsConfigPathRejectsInvalidFilesImmediately .......................... Passed 0.10 sec
Start 262: api.GewuSmartServeApiTest.ListModelsUsesGewuOwnerAndOpenAIShape
262/333 Test #262: api.GewuSmartServeApiTest.ListModelsUsesGewuOwnerAndOpenAIShape ...................................... Passed 0.60 sec
Start 263: api.GewuSmartServeApiTest.GetModelInfoAndDownloadInvalidModel
263/333 Test #263: api.GewuSmartServeApiTest.GetModelInfoAndDownloadInvalidModel ........................................ Passed 0.64 sec
Start 264: api.GewuSmartServeApiTest.ConcurrentListModelsAndErrorStringCallsAreThreadSafe
264/333 Test #264: api.GewuSmartServeApiTest.ConcurrentListModelsAndErrorStringCallsAreThreadSafe ....................... Passed 0.64 sec
Start 265: api.GewuSmartServeApiTest.ErrorStringCoversPublicErrorCodes
265/333 Test #265: api.GewuSmartServeApiTest.ErrorStringCoversPublicErrorCodes .......................................... Passed 0.04 sec
Start 266: api.GewuSmartServeApiTest.EightThreadsInferenceStress
266/333 Test #266: api.GewuSmartServeApiTest.EightThreadsInferenceStress ................................................ Passed 0.60 sec
Start 267: api.GewuSmartServeApiTest.RapidCancelRestartStress
267/333 Test #267: api.GewuSmartServeApiTest.RapidCancelRestartStress ................................................... Passed 0.64 sec
Start 268: api.GewuSmartServeApiTest.ConcurrentCancelOfSameRequest
268/333 Test #268: api.GewuSmartServeApiTest.ConcurrentCancelOfSameRequest .............................................. Passed 0.63 sec
Start 269: api.GewuSmartServeApiTest.ConcurrentGetModelInfo
269/333 Test #269: api.GewuSmartServeApiTest.ConcurrentGetModelInfo ..................................................... Passed 0.64 sec
Start 270: api.GewuSmartServeApiTest.ConcurrentDeleteDuringInference
270/333 Test #270: api.GewuSmartServeApiTest.ConcurrentDeleteDuringInference ............................................ Passed 0.63 sec
Start 271: api.GewuSmartServeApiTest.ConcurrentDownloadDuringInference
271/333 Test #271: api.GewuSmartServeApiTest.ConcurrentDownloadDuringInference ..........................................***Failed 0.64 sec
Running main() from /Users/linhan/repos/ci_smartserve/smartserve/third-party/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = GewuSmartServeApiTest.ConcurrentDownloadDuringInference
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from GewuSmartServeApiTest
[ RUN ] GewuSmartServeApiTest.ConcurrentDownloadDuringInference
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.007 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0 (Apple M4 Pro)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 19069.67 MB
2026-08-19 15:52:10.515 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:10.515 [ WARN][SmartServe][SmartServeAppInit] No models loaded from config
2026-08-19 15:52:10.515 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:10.516 [ WARN][SmartServe][Service] concurrent-stress-engine engine not registered, skipping model: cg-download-on-inference
2026-08-19 15:52:10.516 [ WARN][SmartServe][Service] Model file not found, skipping: cg-download-on-inference (/private/var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/gewu_smartserve_download_on_inference/models/cg-download-on-inference/downloaded.bin)
2026-08-19 15:52:10.516 [ERROR][SmartServe][SmartServeDownloadModel] insecure download source id=cg-download-on-inference url=file:///var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/gewu_smartserve_download_on_inference/source.bin
/Users/linhan/repos/ci_smartserve/smartserve/test/api/gewu_smartserve_concurrent_test.cpp:316: Failure
Expected equality of these values:
GewuSmartServeDownloadModel(modelId.c_str(), nullptr, nullptr)
Which is: -12
GEWU_SMARTSERVE_OK
Which is: 0
[ FAILED ] GewuSmartServeApiTest.ConcurrentDownloadDuringInference (86 ms)
[----------] 1 test from GewuSmartServeApiTest (86 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (86 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] GewuSmartServeApiTest.ConcurrentDownloadDuringInference
1 FAILED TEST
Start 272: api.GewuSmartServeApiTest.RepeatedDownloadIsIdempotent
272/333 Test #272: api.GewuSmartServeApiTest.RepeatedDownloadIsIdempotent ............................................... Passed 0.64 sec
Start 273: api.GewuSmartServeApiTest.SmartServeEngineWrappersRegisterListAndUnregister
273/333 Test #273: api.GewuSmartServeApiTest.SmartServeEngineWrappersRegisterListAndUnregister .......................... Passed 0.64 sec
Start 274: api.GewuSmartServeApiTest.RegisterEngineCanBeUnregistered
274/333 Test #274: api.GewuSmartServeApiTest.RegisterEngineCanBeUnregistered ............................................ Passed 0.64 sec
Start 275: api.GewuSmartServeApiTest.BuiltInEngineCannotBeExternallyRegisteredOrUnregistered
275/333 Test #275: api.GewuSmartServeApiTest.BuiltInEngineCannotBeExternallyRegisteredOrUnregistered .................... Passed 0.64 sec
Start 276: api.GewuSmartServeApiTest.AppleFoundationModelCanBeRegisteredAfterInitializeAndServeChat
276/333 Test #276: api.GewuSmartServeApiTest.AppleFoundationModelCanBeRegisteredAfterInitializeAndServeChat ............. Passed 0.64 sec
Start 277: api.GewuSmartServeApiTest.SmartServeApiRegistersSystemEngineAndServesChat
277/333 Test #277: api.GewuSmartServeApiTest.SmartServeApiRegistersSystemEngineAndServesChat ............................ Passed 0.64 sec
Start 278: api.GewuSmartServeApiTest.AppleFoundationModelHandlesConcurrentRequests
278/333 Test #278: api.GewuSmartServeApiTest.AppleFoundationModelHandlesConcurrentRequests .............................. Passed 0.63 sec
Start 279: api.GewuSmartServeApiTest.AppleFoundationModelStreamingReturnsChunksAndEof
279/333 Test #279: api.GewuSmartServeApiTest.AppleFoundationModelStreamingReturnsChunksAndEof ........................... Passed 0.64 sec
Start 280: api.GewuSmartServeApiTest.CancelActiveRequestReturnsCancelledOnGetResponse
280/333 Test #280: api.GewuSmartServeApiTest.CancelActiveRequestReturnsCancelledOnGetResponse ........................... Passed 0.64 sec
Start 281: api.GewuSmartServeApiTest.DestroyRequestIsSafeWhileGetResponseIsRunning
281/333 Test #281: api.GewuSmartServeApiTest.DestroyRequestIsSafeWhileGetResponseIsRunning .............................. Passed 0.63 sec
Start 282: api.GewuSmartServeApiTest.FinalizeCancelsActiveRequestAndClearsOpaqueHandle
282/333 Test #282: api.GewuSmartServeApiTest.FinalizeCancelsActiveRequestAndClearsOpaqueHandle .......................... Passed 0.64 sec
Start 283: api.GewuSmartServeApiTest.CancelCompletedRequestReturnsOk
283/333 Test #283: api.GewuSmartServeApiTest.CancelCompletedRequestReturnsOk ............................................ Passed 0.63 sec
Start 284: api.GewuSmartServeApiTest.StreamingRequestCanBeCancelledAfterFirstChunk
284/333 Test #284: api.GewuSmartServeApiTest.StreamingRequestCanBeCancelledAfterFirstChunk .............................. Passed 0.64 sec
Start 285: api.GewuSmartServeApiTest.UnloadReturnsBusyAndDeleteRejectsSystemModelWhileInferenceIsActive
285/333 Test #285: api.GewuSmartServeApiTest.UnloadReturnsBusyAndDeleteRejectsSystemModelWhileInferenceIsActive ......... Passed 0.64 sec
Start 286: api.GewuSmartServeApiTest.DeleteSystemModelDoesNotRemoveSameNamedDirectory
286/333 Test #286: api.GewuSmartServeApiTest.DeleteSystemModelDoesNotRemoveSameNamedDirectory ........................... Passed 0.62 sec
Start 287: api.GewuSmartServeApiTest.InferenceRejectsConfiguredModelWithMissingDeclaredFiles
287/333 Test #287: api.GewuSmartServeApiTest.InferenceRejectsConfiguredModelWithMissingDeclaredFiles .................... Passed 0.63 sec
Start 288: api.GewuSmartServeApiTest.AppleFoundationModelLoadsFromConfigPathAfterInitializeAndRegister
288/333 Test #288: api.GewuSmartServeApiTest.AppleFoundationModelLoadsFromConfigPathAfterInitializeAndRegister .......... Passed 0.64 sec
Start 289: api.GewuSmartServeApiTest.UnregisterBusyEngineThenRetrySucceeds
289/333 Test #289: api.GewuSmartServeApiTest.UnregisterBusyEngineThenRetrySucceeds ...................................... Passed 0.63 sec
Start 290: api.GewuSmartServeApiTest.UnknownModelIdsRejectedByModelApis
290/333 Test #290: api.GewuSmartServeApiTest.UnknownModelIdsRejectedByModelApis ......................................... Passed 0.63 sec
Start 291: api.GewuSmartServeApiTest.ConfiguredModelWithUnsafeStoragePathRejected
291/333 Test #291: api.GewuSmartServeApiTest.ConfiguredModelWithUnsafeStoragePathRejected ............................... Passed 0.63 sec
Start 292: api.GewuSmartServeApiTest.PreinstalledModelWithoutVerifiedDownloadIsNotLoadable
292/333 Test #292: api.GewuSmartServeApiTest.PreinstalledModelWithoutVerifiedDownloadIsNotLoadable ......................***Skipped 0.04 sec
Start 293: api.GewuSmartServeApiTest.MultiFileDownloadReportsInstalledFileProgressProportionally
293/333 Test #293: api.GewuSmartServeApiTest.MultiFileDownloadReportsInstalledFileProgressProportionally ................ Passed 0.61 sec
Start 294: api.GewuSmartServeApiTest.SingleFileDownloadUsesNameInsideModelPathDirectory
294/333 Test #294: api.GewuSmartServeApiTest.SingleFileDownloadUsesNameInsideModelPathDirectory ......................... Passed 0.64 sec
Start 295: api.GewuSmartServeApiTest.DownloadAcceptsCustomHttpsAndRejectsInsecureSources
295/333 Test #295: api.GewuSmartServeApiTest.DownloadAcceptsCustomHttpsAndRejectsInsecureSources ........................ Passed 0.64 sec
Start 296: api.GewuSmartServeApiTest.SingleFileDirectoryPathIsNotReportedDownloaded
296/333 Test #296: api.GewuSmartServeApiTest.SingleFileDirectoryPathIsNotReportedDownloaded ............................. Passed 0.64 sec
Start 297: api.GewuSmartServeApiTest.MultiFileModelRequiresValidChecksumsToBeDownloaded
297/333 Test #297: api.GewuSmartServeApiTest.MultiFileModelRequiresValidChecksumsToBeDownloaded ......................... Passed 0.63 sec
Start 298: api.GewuSmartServeApiTest.DownloadRejectsPathTraversalInModelAndFilePaths
298/333 Test #298: api.GewuSmartServeApiTest.DownloadRejectsPathTraversalInModelAndFilePaths ............................ Passed 0.63 sec
Start 299: api.GewuSmartServeApiTest.ServiceLoadModelsUsesSingleFileNameInsideModelPathDirectory
299/333 Test #299: api.GewuSmartServeApiTest.ServiceLoadModelsUsesSingleFileNameInsideModelPathDirectory ................ Passed 0.63 sec
Start 300: api.GewuSmartServeApiTest.ServiceLoadModelsRejectsDirectoryForSingleFileModel
300/333 Test #300: api.GewuSmartServeApiTest.ServiceLoadModelsRejectsDirectoryForSingleFileModel ........................ Passed 0.63 sec
Start 301: api.GewuSmartServeApiTest.ServiceLoadModelsRejectsEscapedConfiguredPaths
301/333 Test #301: api.GewuSmartServeApiTest.ServiceLoadModelsRejectsEscapedConfiguredPaths ............................. Passed 0.63 sec
Start 302: api.GewuSmartServeApiTest.DownloadPersistsMetadataAndDeleteCleansIt
302/333 Test #302: api.GewuSmartServeApiTest.DownloadPersistsMetadataAndDeleteCleansIt .................................. Passed 0.64 sec
Start 303: api.GewuSmartServeApiTest.PauseDownloadCancelsAndNextDownloadResumes
303/333 Test #303: api.GewuSmartServeApiTest.PauseDownloadCancelsAndNextDownloadResumes ................................. Passed 0.65 sec
Start 304: api.GewuSmartServeApiTest.ProgressCallbackRejectsBlockingLifecycleReentryWithoutDeadlock
304/333 Test #304: api.GewuSmartServeApiTest.ProgressCallbackRejectsBlockingLifecycleReentryWithoutDeadlock ............. Passed 0.63 sec
Start 305: api.GewuSmartServeApiTest.ThreeConcurrentDownloadsOfSameModelAreIdempotent
305/333 Test #305: api.GewuSmartServeApiTest.ThreeConcurrentDownloadsOfSameModelAreIdempotent ........................... Passed 0.64 sec
Start 306: api.GewuSmartServeApiTest.DeleteDuringDownloadCancelsAndRemovesPartialModel
306/333 Test #306: api.GewuSmartServeApiTest.DeleteDuringDownloadCancelsAndRemovesPartialModel .......................... Passed 0.65 sec
Start 307: api.GewuSmartServeApiTest.DownloadStartedDuringDeleteWaitsAndRestoresModel
307/333 Test #307: api.GewuSmartServeApiTest.DownloadStartedDuringDeleteWaitsAndRestoresModel ........................... Passed 0.64 sec
Start 308: api.GewuSmartServeApiTest.InferenceStartedDuringDeleteIsRejected
308/333 Test #308: api.GewuSmartServeApiTest.InferenceStartedDuringDeleteIsRejected ..................................... Passed 0.63 sec
Start 309: api.GewuSmartServeApiTest.ConcurrentRepeatedDeleteHasDeterministicNotFoundResult
309/333 Test #309: api.GewuSmartServeApiTest.ConcurrentRepeatedDeleteHasDeterministicNotFoundResult ..................... Passed 0.64 sec
Start 310: api.GewuSmartServeApiTest.ModelQueriesDoNotWaitForActiveDownload
310/333 Test #310: api.GewuSmartServeApiTest.ModelQueriesDoNotWaitForActiveDownload ..................................... Passed 0.63 sec
Start 311: api.GewuSmartServeApiTest.ModelQueriesReturnBusyDuringDelete
311/333 Test #311: api.GewuSmartServeApiTest.ModelQueriesReturnBusyDuringDelete ......................................... Passed 0.64 sec
Start 312: api.GewuSmartServeApiTest.DeletingLargeModelDoesNotBlockUnrelatedEngineRegistration
312/333 Test #312: api.GewuSmartServeApiTest.DeletingLargeModelDoesNotBlockUnrelatedEngineRegistration .................. Passed 1.97 sec
Start 313: api.GewuSmartServeApiTest.LoadModelsSkipsModelWhileItsFilesAreBeingDeleted
313/333 Test #313: api.GewuSmartServeApiTest.LoadModelsSkipsModelWhileItsFilesAreBeingDeleted ........................... Passed 0.63 sec
Start 314: api.GewuSmartServeApiTest.DeleteUnloadsRuntimeModelWhenFilesAndMetadataAreAlreadyMissing
314/333 Test #314: api.GewuSmartServeApiTest.DeleteUnloadsRuntimeModelWhenFilesAndMetadataAreAlreadyMissing ............. Passed 0.63 sec
Start 315: api.GewuSmartServeApiTest.PreinstalledModelRequiresMatchingChecksum
315/333 Test #315: api.GewuSmartServeApiTest.PreinstalledModelRequiresMatchingChecksum .................................. Passed 0.64 sec
Start 316: api.GewuSmartServeApiTest.PreinstalledModelWithoutChecksumIsRejected
316/333 Test #316: api.GewuSmartServeApiTest.PreinstalledModelWithoutChecksumIsRejected ................................. Passed 0.63 sec
Start 317: api.GewuSmartServeApiTest.DeleteRejectsIntermediateSymlinkEscapingModelRoot
317/333 Test #317: api.GewuSmartServeApiTest.DeleteRejectsIntermediateSymlinkEscapingModelRoot .......................... Passed 0.64 sec
Start 318: functional.GewuSmartServeApiTest.FunctionalLifecycleSmoke
318/333 Test #318: functional.GewuSmartServeApiTest.FunctionalLifecycleSmoke ............................................ Passed 0.63 sec
Start 319: functional.GewuSmartServeApiTest.FunctionalEngineRegistrationSmoke
319/333 Test #319: functional.GewuSmartServeApiTest.FunctionalEngineRegistrationSmoke ................................... Passed 0.64 sec
Start 320: functional.GewuSmartServeApiTest.FunctionalModelConfigSmoke
320/333 Test #320: functional.GewuSmartServeApiTest.FunctionalModelConfigSmoke .......................................... Passed 0.64 sec
Start 321: functional.GewuSmartServeApiTest.FunctionalNonStreamingChatSmoke
321/333 Test #321: functional.GewuSmartServeApiTest.FunctionalNonStreamingChatSmoke ..................................... Passed 0.63 sec
Start 322: functional.GewuSmartServeApiTest.FunctionalStreamingChatSmoke
322/333 Test #322: functional.GewuSmartServeApiTest.FunctionalStreamingChatSmoke ........................................ Passed 0.64 sec
Start 323: functional.GewuSmartServeApiTest.FunctionalCancelRequestSmoke
323/333 Test #323: functional.GewuSmartServeApiTest.FunctionalCancelRequestSmoke ........................................ Passed 0.63 sec
Start 324: functional.GewuSmartServeApiTest.FunctionalSystemModelSmoke
324/333 Test #324: functional.GewuSmartServeApiTest.FunctionalSystemModelSmoke .......................................... Passed 0.64 sec
Start 325: functional.GewuSmartServeApiTest.FunctionalEngineSubmitFailureSmoke
325/333 Test #325: functional.GewuSmartServeApiTest.FunctionalEngineSubmitFailureSmoke .................................. Passed 0.63 sec
Start 326: functional.GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
326/333 Test #326: functional.GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke ......................................***Failed 0.64 sec
Running main() from /Users/linhan/repos/ci_smartserve/smartserve/third-party/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from GewuSmartServeApiTest
[ RUN ] GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.007 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0 (Apple M4 Pro)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 19069.67 MB
2026-08-19 15:52:46.228 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:46.228 [ WARN][SmartServe][SmartServeAppInit] No models loaded from config
2026-08-19 15:52:46.228 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:46.228 [ WARN][SmartServe][Service] Model file not found, skipping: func-download-model (/private/var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/func_download_smoke/models/func-download-model/downloaded.bin)
2026-08-19 15:52:46.228 [ WARN][SmartServe][Service] Model file not found, skipping: func-download-model-b (/private/var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/func_download_smoke/models/func-download-model-b/second.bin)
2026-08-19 15:52:46.228 [ERROR][SmartServe][SmartServeDownloadModel] insecure download source id=func-download-model url=file:///var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/func_download_smoke/source.bin
/Users/linhan/repos/ci_smartserve/smartserve/test/functional/functional_smoke_test.cpp:400: Failure
Expected equality of these values:
GewuSmartServeDownloadModel(modelId.c_str(), CaptureFirstProgress, &capture)
Which is: -12
GEWU_SMARTSERVE_OK
Which is: 0
/Users/linhan/repos/ci_smartserve/smartserve/test/functional/functional_smoke_test.cpp:401: Failure
Expected: (capture.calls.load()) > (0), actual: 0 vs 0
/Users/linhan/repos/ci_smartserve/smartserve/test/functional/functional_smoke_test.cpp:402: Failure
Value of: std::filesystem::is_regular_file(destPath)
Actual: false
Expected: true
[ FAILED ] GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke (88 ms)
[----------] 1 test from GewuSmartServeApiTest (88 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (88 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
1 FAILED TEST
Start 327: functional.GewuSmartServeApiTest.FunctionalDeleteDuringDownloadSmoke
327/333 Test #327: functional.GewuSmartServeApiTest.FunctionalDeleteDuringDownloadSmoke ................................. Passed 0.65 sec
Start 328: model_runtime.MultiSessionTest.MultiSessionAndSingleSessionMultiTask
328/333 Test #328: model_runtime.MultiSessionTest.MultiSessionAndSingleSessionMultiTask .................................***Skipped 0.64 sec
Start 329: model_runtime.MultiSessionTest.LlamaCppHonorsRequestOutputLimit
329/333 Test #329: model_runtime.MultiSessionTest.LlamaCppHonorsRequestOutputLimit ......................................***Skipped 0.63 sec
Start 330: model_runtime.MultiSessionTest.RuntimeCancelThenImmediateNewRequest
330/333 Test #330: model_runtime.MultiSessionTest.RuntimeCancelThenImmediateNewRequest ..................................***Skipped 0.63 sec
Start 331: model_runtime.SmartServeTest.SmartServeLLMInterfaceTest
331/333 Test #331: model_runtime.SmartServeTest.SmartServeLLMInterfaceTest ..............................................***Skipped 0.63 sec
Start 332: model_runtime.SmartServeTest.SmartServeMnnLoraInterfaceTest
332/333 Test #332: model_runtime.SmartServeTest.SmartServeMnnLoraInterfaceTest ..........................................***Skipped 0.64 sec
Start 333: model_runtime.SmartServeTest.SmartServeVLMInterfaceTest
333/333 Test #333: model_runtime.SmartServeTest.SmartServeVLMInterfaceTest ..............................................***Skipped 0.64 sec
99% tests passed, 2 tests failed out of 333
Total Test time (real) = 61.66 sec
The following tests did not run:
242 - api.GewuSmartServeApiTest.ModelsDirectoryRejectsGroupOrWorldWritableRoot (Skipped)
292 - api.GewuSmartServeApiTest.PreinstalledModelWithoutVerifiedDownloadIsNotLoadable (Skipped)
328 - model_runtime.MultiSessionTest.MultiSessionAndSingleSessionMultiTask (Skipped)
329 - model_runtime.MultiSessionTest.LlamaCppHonorsRequestOutputLimit (Skipped)
330 - model_runtime.MultiSessionTest.RuntimeCancelThenImmediateNewRequest (Skipped)
331 - model_runtime.SmartServeTest.SmartServeLLMInterfaceTest (Skipped)
332 - model_runtime.SmartServeTest.SmartServeMnnLoraInterfaceTest (Skipped)
333 - model_runtime.SmartServeTest.SmartServeVLMInterfaceTest (Skipped)
The following tests FAILED:
271 - api.GewuSmartServeApiTest.ConcurrentDownloadDuringInference (Failed)
326 - functional.GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke (Failed)
Errors while running CTest
[1;33mSome tests failed; rerunning failed tests once...[0m
Test project /Users/linhan/repos/ci_smartserve/smartserve/build/macos
Start 271: api.GewuSmartServeApiTest.ConcurrentDownloadDuringInference
1/2 Test #271: api.GewuSmartServeApiTest.ConcurrentDownloadDuringInference .......***Failed 0.61 sec
Running main() from /Users/linhan/repos/ci_smartserve/smartserve/third-party/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = GewuSmartServeApiTest.ConcurrentDownloadDuringInference
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from GewuSmartServeApiTest
[ RUN ] GewuSmartServeApiTest.ConcurrentDownloadDuringInference
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.007 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0 (Apple M4 Pro)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 19069.67 MB
2026-08-19 15:52:51.338 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:51.338 [ WARN][SmartServe][SmartServeAppInit] No models loaded from config
2026-08-19 15:52:51.338 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:51.339 [ WARN][SmartServe][Service] concurrent-stress-engine engine not registered, skipping model: cg-download-on-inference
2026-08-19 15:52:51.339 [ WARN][SmartServe][Service] Model file not found, skipping: cg-download-on-inference (/private/var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/gewu_smartserve_download_on_inference/models/cg-download-on-inference/downloaded.bin)
2026-08-19 15:52:51.339 [ERROR][SmartServe][SmartServeDownloadModel] insecure download source id=cg-download-on-inference url=file:///var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/gewu_smartserve_download_on_inference/source.bin
/Users/linhan/repos/ci_smartserve/smartserve/test/api/gewu_smartserve_concurrent_test.cpp:316: Failure
Expected equality of these values:
GewuSmartServeDownloadModel(modelId.c_str(), nullptr, nullptr)
Which is: -12
GEWU_SMARTSERVE_OK
Which is: 0
[ FAILED ] GewuSmartServeApiTest.ConcurrentDownloadDuringInference (79 ms)
[----------] 1 test from GewuSmartServeApiTest (79 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (79 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] GewuSmartServeApiTest.ConcurrentDownloadDuringInference
1 FAILED TEST
Start 326: functional.GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
2/2 Test #326: functional.GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke ...***Failed 0.64 sec
Running main() from /Users/linhan/repos/ci_smartserve/smartserve/third-party/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from GewuSmartServeApiTest
[ RUN ] GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 0.007 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name: MTL0 (Apple M4 Pro)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple9 (1009)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4 (5002)
ggml_metal_device_init: simdgroup reduction = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory = true
ggml_metal_device_init: has bfloat = true
ggml_metal_device_init: has tensor = false
ggml_metal_device_init: use residency sets = true
ggml_metal_device_init: use shared buffers = true
ggml_metal_device_init: recommendedMaxWorkingSetSize = 19069.67 MB
2026-08-19 15:52:51.981 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:51.981 [ WARN][SmartServe][SmartServeAppInit] No models loaded from config
2026-08-19 15:52:51.981 [ERROR][SmartServe][Service] MODEL_CONFIG_PATH is not configured; keeping the previous model state
2026-08-19 15:52:51.981 [ WARN][SmartServe][Service] Model file not found, skipping: func-download-model (/private/var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/func_download_smoke/models/func-download-model/downloaded.bin)
2026-08-19 15:52:51.982 [ WARN][SmartServe][Service] Model file not found, skipping: func-download-model-b (/private/var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/func_download_smoke/models/func-download-model-b/second.bin)
2026-08-19 15:52:51.982 [ERROR][SmartServe][SmartServeDownloadModel] insecure download source id=func-download-model url=file:///var/folders/70/p5lhbxd505j8tnl4nkn1lp800000gn/T/func_download_smoke/source.bin
/Users/linhan/repos/ci_smartserve/smartserve/test/functional/functional_smoke_test.cpp:400: Failure
Expected equality of these values:
GewuSmartServeDownloadModel(modelId.c_str(), CaptureFirstProgress, &capture)
Which is: -12
GEWU_SMARTSERVE_OK
Which is: 0
/Users/linhan/repos/ci_smartserve/smartserve/test/functional/functional_smoke_test.cpp:401: Failure
Expected: (capture.calls.load()) > (0), actual: 0 vs 0
/Users/linhan/repos/ci_smartserve/smartserve/test/functional/functional_smoke_test.cpp:402: Failure
Value of: std::filesystem::is_regular_file(destPath)
Actual: false
Expected: true
[ FAILED ] GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke (87 ms)
[----------] 1 test from GewuSmartServeApiTest (87 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (87 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke
1 FAILED TEST
0% tests passed, 2 tests failed out of 2
Total Test time (real) = 1.27 sec
The following tests FAILED:
271 - api.GewuSmartServeApiTest.ConcurrentDownloadDuringInference (Failed)
326 - functional.GewuSmartServeApiTest.FunctionalDownloadCleanupSmoke (Failed)
Errors while running CTest
=== finished status=failure exit=8 duration=263s timeout_sec=900 attempts=2 ===