Severe Thread Lag

[John Adams]

Faux and I had a confusing discussion about threading the other night, and it brought up a pain point we've been working around until now. We need to evaluate and fix any of our "main process thread delays".

Just witnessed this on NT, thankfully Sile actually mentioned the lag (people normally don't).
[quote]16:29:27.446 D Chat [NewTelon] Moldew: Hush you!
16:30:03.185 D Chat [Say] Sile : hmm
16:30:14.510 D Vendor ClientMerchantRequest activated.
16:30:14.510 D Vendor ClientRepairMerchant creating vendor.
16:30:14.510 D Char Setting vendor to character Sandiinya Moonshadow
16:30:14.510 D Vendor Vendor::CreateInventory Creating Vendor Inventory For (Crafting Instructor).
16:30:14.510 D Vendor Vendor::SendInventoryToClient Sending Vendor Inventory.
16:30:21.062 D Char Setting vendor to character Sandiinya Moonshadow
16:30:23.621 D Chat [NewTelon] Sile: Server ate my message? D=
16:30:28.956 D Chat [NewTelon] Sile: ah there we go, that fixed it
16:30:37.535 D Vendor ClientRepairMerchant activated.
16:30:37.535 D Vendor ClientRepairMerchant creating vendor.
16:30:37.535 D Char Setting vendor to character Sandiinya Moonshadow
16:30:37.535 D Vendor Vendor::OpenRepairMerchantWindow: Sending character item unique_id 58, durability 85 to repair window.
16:30:37.535 D Vendor Vendor::OpenRepairMerchantWindow: Sending character item unique_id 53, durability 85 to repair window.
16:30:42.730 D Vendor ClientRepairAll activated.
16:30:42.730 D Vendor Vendor::RepairAll: Repairing item unique_id 58.
16:30:42.730 D Vendor Vendor::RepairAll: Repairing item unique_id 53.
16:30:45.148 D Char Setting vendor to character Sandiinya Moonshadow
16:30:48.907 D Chat [NewTelon] Sile: That was a major delay there[/quote]
This is just one example of many where Chatting gets disrupted by some other action taking place anywhere else in the world. I do not consider this "performance enhancements" for later. This should not be happening >today< and the more we understand why it's happening, the less it will happen.

I'd like someone to take a look into it, maybe reproduce it on their local server with 2 characters, one spamming chat macros while the other opens a merchant window... and the other known delay is summoning a mount. There is literally a 5s delay in the whole world when someone next to me pops up their flying mount.

This is mostly an inconvenience right now, but there's a root cause we all need to understand, and stop the behavior before it gets worse. For all I know, it's been there since we had 1 thread, and seems related to World communications... a place where we have many, many issues still.

Thanks

[Moldew]

Thank you for bringing it up! Been an annoyance for a while now...

Bugs 194 and 205: viewtopic.php?f=22&t=1508&p=14029&hilit=delay#p14029

Mentioned it in IRC buncha times, but people get scared of me there...I don't know why!! (cracking down on yous!!!)

[zippyzee]

Looking at that example there are parts clustered together in vendor that all fire off together. The time spacing in between would be while the player is navigating buttons, etc. and there shouldn't be anything computationally going on. Unless we can see it duplicated surrounding particular vendor operations. If so I'll have to do some timing tests in those sections.

Mounting/dismounting is certainly not a final implementation but I'm not sure why it would hog any server time. That we definitely need to look into.

If it runs deeper than that, then I don't know what the hiccup is from. The only lag I see on my server is during the save routine. I'm of course never chatting, and nobody is firing up mounts nearby, and I have a much lighter load.

There is no code that I can think of that I've written that has any sort of intentional delay that would hold up a thread. Plenty that is computationally intense, but none that wait for any reason.

Certainly we need to identify things like this ASAP.

[John Adams]

I also noticed last night that when Sile did a .goto John, as he was arriving above my head there was about a 3-4s delay where my client completely froze up, even flashed (Not Responding) while his flying-mounted character appeared over my head.

I definitely know mounts were "hacked in for fun", and not a final implementation. I only mentioned it because it could now be turned into an actual spell summoning properly, if the current code is causing huge packet spikes to send the initial data to the surrounding clients. Not that doing it "proper" will result in anything less spikey...