New Telons New Behavior
Posted: Tue Nov 10, 2015 4:42 am
The last couple of outages for NT have been no crash, simply hung server processes. The code, from a console perspective, is running fine. Last night, looks like there were 3 clients across 2 chunks. At around 19:30 hours, a mutex error was thrown (this is the primary symptom each time)
[quote]19:22:38.510 D Chunk Camp started for character 'Schiller'
19:22:58.774 D Chunk Camping out character 'Schiller'
19:22:58.774 D Char ToggleIsOnline: Setting player offline
19:22:58.790 D Char No abilities need saving for 'Schiller' (356)
19:22:58.790 D Char Saved successful for character 'Schiller' (356)
19:22:58.790 D Guilds Schiller is not in a guild.
19:22:58.805 D Char ToggleIsOnline: Setting player offline
19:22:58.805 I UDP Client from xxx:52135 set to disconnect : Application
19:23:01.816 I UDP Client from xxx:52135 has been removed.
19:23:01.863 I Chunk Starting chunk shutdown timer for Plains of Anguish.
19:24:10.423 I UDP Client from xxx:58685 set to disconnect : Timeout
19:24:12.451 I UDP Client from xxx:58685 has been removed.
19:24:15.774 I UDP New client connected from xxx:60712
19:24:15.774 I UDP Received session request from xxx:60712 with connection ID 169018090
19:25:25.457 I UDP Client from xxx:60712 set to disconnect : Timeout
19:25:27.485 I UDP Client from xxx:60712 has been removed.
19:28:01.888 I Chunk Shutdown timer expired. Shutting down Plains of Anguish.
19:28:11.903 E Mutex Timeout trying to acquire mutex ChunkServerList::remove_chunk_s's write lock lock after 10000 milliseconds[/quote]
The last player logged off shortly after, making me think the chunk he was in was still running fine:
[quote]19:40:42.804 D Chunk Camp started for character 'Theun'
19:41:03.333 D Chunk Camping out character 'Theun'
19:41:03.333 D Char ToggleIsOnline: Setting player offline
19:41:03.333 D Char No abilities need saving for 'Theun' (110)
19:41:03.504 D Char Saved successful for character 'Theun' (110)
19:41:03.520 D Guilds Theun is not in a guild.
19:41:03.520 D Char ToggleIsOnline: Setting player offline
19:41:03.520 I UDP Client from xxx:59057 set to disconnect : Application
19:41:06.390 I UDP Client from xxx:59057 has been removed.
19:41:11.554 I Chunk Starting chunk shutdown timer for Khal.
19:41:45.966 I UDP Client from xxx:55790 set to disconnect : Timeout
19:41:47.994 I UDP Client from xxx:55790 has been removed.
19:46:11.561 I Chunk Shutdown timer expired. Shutting down Khal.[/quote]
And that was all she wrote. New Telon, according to LoginServer, was DOWN. Although from my end, no crash. Seems like the server got hung up in whatever the mutex was and would not accept anymore incoming packets of any kind, making Login think it was offline.
Sadly, this never happens when I am sitting there logged in, but always 100% of the time, the moment I turn off the monitor and walk away for the evening. Sorry to those players trying to check the game out... but there's no way to auto-restart of the world server doesn't crash
[quote]19:22:38.510 D Chunk Camp started for character 'Schiller'
19:22:58.774 D Chunk Camping out character 'Schiller'
19:22:58.774 D Char ToggleIsOnline: Setting player offline
19:22:58.790 D Char No abilities need saving for 'Schiller' (356)
19:22:58.790 D Char Saved successful for character 'Schiller' (356)
19:22:58.790 D Guilds Schiller is not in a guild.
19:22:58.805 D Char ToggleIsOnline: Setting player offline
19:22:58.805 I UDP Client from xxx:52135 set to disconnect : Application
19:23:01.816 I UDP Client from xxx:52135 has been removed.
19:23:01.863 I Chunk Starting chunk shutdown timer for Plains of Anguish.
19:24:10.423 I UDP Client from xxx:58685 set to disconnect : Timeout
19:24:12.451 I UDP Client from xxx:58685 has been removed.
19:24:15.774 I UDP New client connected from xxx:60712
19:24:15.774 I UDP Received session request from xxx:60712 with connection ID 169018090
19:25:25.457 I UDP Client from xxx:60712 set to disconnect : Timeout
19:25:27.485 I UDP Client from xxx:60712 has been removed.
19:28:01.888 I Chunk Shutdown timer expired. Shutting down Plains of Anguish.
19:28:11.903 E Mutex Timeout trying to acquire mutex ChunkServerList::remove_chunk_s's write lock lock after 10000 milliseconds[/quote]
The last player logged off shortly after, making me think the chunk he was in was still running fine:
[quote]19:40:42.804 D Chunk Camp started for character 'Theun'
19:41:03.333 D Chunk Camping out character 'Theun'
19:41:03.333 D Char ToggleIsOnline: Setting player offline
19:41:03.333 D Char No abilities need saving for 'Theun' (110)
19:41:03.504 D Char Saved successful for character 'Theun' (110)
19:41:03.520 D Guilds Theun is not in a guild.
19:41:03.520 D Char ToggleIsOnline: Setting player offline
19:41:03.520 I UDP Client from xxx:59057 set to disconnect : Application
19:41:06.390 I UDP Client from xxx:59057 has been removed.
19:41:11.554 I Chunk Starting chunk shutdown timer for Khal.
19:41:45.966 I UDP Client from xxx:55790 set to disconnect : Timeout
19:41:47.994 I UDP Client from xxx:55790 has been removed.
19:46:11.561 I Chunk Shutdown timer expired. Shutting down Khal.[/quote]
And that was all she wrote. New Telon, according to LoginServer, was DOWN. Although from my end, no crash. Seems like the server got hung up in whatever the mutex was and would not accept anymore incoming packets of any kind, making Login think it was offline.
Sadly, this never happens when I am sitting there logged in, but always 100% of the time, the moment I turn off the monitor and walk away for the evening. Sorry to those players trying to check the game out... but there's no way to auto-restart of the world server doesn't crash