First Signs of Life

[John Adams]

After spending a few days inspecting the data coming back from the UnrealBitLib.dll provided by Xen, I ran my first (of many) auto-parsings last night. When I woke up this morning, it was done - with only 1 error The results were pretty impressive, though they require deeper analysis. Here's some stats -

Logs Parsed:       313
Chunks discovered: 124
Actor entities:    13,282
Unique Placements: 330,723

"Actors" are the in-game entities such as NPCs, mobs, corpses, in-game objects, etc
"Unique Placements" are the individual occurrences of the Actor in a Chunk at an X/Y/Z vector.

The placements are likely duplicates of the same entity whose positions are slightly different (wanderers?) therefore unique, and is more than likely a highly exaggerated number - I don't think the entirety of Telon has 330k spawns but further discovery should help identify just what we have in our inventory.

I plan to spend the day exposing this data to you to help analyze, and working on parsing the Appearances and Attachments data.

If I haven't thanked our collectors enough, let me thank you once again. This is just the beginning of data parsing, and it's looking amazing.


Edit: Here's a quick view per chunk of placements parsed - you can see why I think they are valid, but exaggerated -

1716		Afrit (71)
2982		Ahgram (74)
3393		Ancient Port Warehouse (371)
3455		Beranid Hills (127)
2		Bley's Bluff (260)
3922		Blighted Lands (29)
7487		Bordinar's Cleft (136)
12757		Ca'ial Brael (22)
6		Ceros Isle (16)
1		Ceros Isle (26)
200		Ceros Isle (25)
18026		Cliffs of Ghelgad (86)
688		Coastal Graveyard (154)
25		Coterie Infinium Sanctuary (66)
140		Cragwind Ridge (58)
25		Daegarmo Isle (35)
1117		Dragon's Backbone (53)
87		Dragon's Maw (39)
888		Falgarholm (175)
42		Fathesi Steppe (61)
61		Fields of Despair (128)
244		Flordiel (133)
4477		Grimsea Watch (11)
3803		Hag's Coastline (77)
4932		Hathor Zhi (67)
99		Heroes' Plain (121)
274		Innovus (123)
1401		Islands of Madness (46)
17248		Isle of Dawn (1)
27		Isle of Man (337)
4985		Jalen's Crossing (23)
25		Jalen's Retreat (41)
746		Jathred's Twist (83)
2293		Jharru Flats (84)
287		Kaon's Rush (166)
1		Karrus Hakrel (92)
34710		Khal (85)
3		Kojan Depths (6)
1		Kojan Shores (315)
368		Ksaravi Gulch (94)
15668		Leth Nurae (159)
89		Lilly's Sanctuary (366)
15166		Lomshir (68)
15603		Lomshir Plain (76)
5488		Magi Hold (24)
10248		Martok (10)
314		Martok Coast (20)
1		Martok Sea (18)
2		Martok Sea (27)
83		Mekalia (100)
85		Mind's Crown (55)
7251		Misthaven Crossing (168)
3683		New Targonor (141)
31		Northern Stonepine Ridge (188)
10717		Nusibe Necropolis (51)
2634		Old Targonor (122)
253		Pankor Zhi (60)
2831		Plains of Anquish (129)
26		Port Aughos (2)
1714		Qa Riverbank (93)
7		Qalian Sea (47)
17682		Rahz Inkur (72)
258		Razad (73)
1377		Renton Keep (152)
3397		River Palace (52)
1595		River Valley (59)
112		Rridge of Memories (137)
164		Ruin Falls (126)
4782		Ruins of Trengal Keep (153)
34		Ruins of Vol Tuniel (172)
383		Sands of Sorrow (81)
72		Sea of Kojan (7)
15		Sea of Kojan (4)
6		Seas of Mist (330)
10		Seas of Mist (316)
2138		Seawatch Coast (45)
1975		Seawatch Cove (44)
384		Sepulchral Chasm (82)
1404		Shang Village (15)
930		Shimmering Shallows (70)
5		Shrine of the Flame (142)
5025		Silverlake (160)
1544		Skawlra Rock (102)
1137		Skrilien Point (105)
2		Southwatch (135)
1192		Strand of the Ancients (104)
774		Stranglewater Shore (54)
280		Sundering Wastelands (19)
1129		Sunset Pointe (40)
98		Swamps of Rumug (124)
15611		Tanvu (14)
4		Tanvu Bay (13)
852		Tar Janashir (91)
238		Tauthien Delta (101)
2598		Tawar Galan (12)
99		Tehatamani Harbor (49)
107		Temple of Tehatamani (50)
7		The Deebs (161)
65		The Temple of Dailuk (75)
7077		The Tomb of Lord Tsang (30)
9371		Three Rivers (143)
10456		Tursh Village (144)
19		Unknown (303)
9400		Vault of Heroes (174)
1429		Veskal's Exchange (167)
228		Vsurog Moor (125)
181		Wardship of the Sleeping Moon (173)
172		Wildgrowth Forest (9)
67		Wyrmwing Slopes (43)

[Xinux]

And then there was life. Nice work

[John Adams]

Thanks. While this is mostly good news, there is some bad news. Above I said I parsed 313 logs, though I just discovered I have 400 - that means 87 logs did not find a single NPC. After running through a few manually, the bsList (Xen-speak) has the packets, but there are no detectable SGONPCPawn objects.

Xen was looking into this for some sample logs I provided, so when he figures out what's going on in UnrealBitLib, the NPC totals will surely increase.

[Zewtastic]

Many may have the fact that the collector has been updated numerous times, if that has any impact.

[John Adams]

[quote="zingtastic"]Many may have the fact that the collector has been updated numerous times, if that has any impact.[/quote]
No, the changes to the collector have rarely been about capturing spawn data. It's likely something we just do not know about the bitstream yet - or the fact that Xinux told folks to minimize changing chunks, and you know how that went Though the latter would not render an entire log bogus, so I'm going with the former.

[Xinux]

Yea updates to the collector were just opcode updates and misc updates nothing that would effect spawn collects.

[Xen]

[quote="John Adams"]Thanks. While this is mostly good news, there is some bad news. Above I said I parsed 313 logs, though I just discovered I have 400 - that means 87 logs did not find a single NPC. After running through a few manually, the bsList (Xen-speak) has the packets, but there are no detectable SGONPCPawn objects.

Xen was looking into this for some sample logs I provided, so when he figures out what's going on in UnrealBitLib, the NPC totals will surely increase.[/quote]

John, can you give me a couple names of logs that did not find an NPC? I'll run through them and see what is going on.

[John Adams]

DerogarsOutpost_Level1-5.log		JohnAdams
Halgarad_MerchantsConvosQuests.log		JohnAdams
Halgarad_Quests5-9.log		JohnAdams
Halgarad_Level10.log		JohnAdams
Halgarad_Level11-See_Notes.log		JohnAdams
ThreeRivers_20140307d.log		jududdar
Dahknarg_20140205a.log		Swampdog
Dahknarg_20140206a.log		Swampdog
Dahknarg_20140206b.log		Swampdog
Dahknarg_20140206c.log		Swampdog
Dahknarg_20140206d.log		Swampdog
Dahknarg_20140207a.log		Swampdog
Dahknarg_20140207b.log		Swampdog
Dahknarg_20140208a_BloodhowlersCave.log		Swampdog
Dahknarg_20140208b_Crafting_ArtificierOutfitter.log		Swampdog
Dahknarg_20140208c_Crafting_SmithQuests.log		Swampdog
Dahknarg_20140209a_CraftingOutfitter.log		Swampdog
Dahknarg_20140210a_CraftingOutfitter.log		Swampdog
Dahknarg_20140210b_CraftingOutfitter.log		Swampdog
Dahknarg_20140212a_Exploration.log		Swampdog
Dahknarg_20140212b_Exploration.log		Swampdog
Dahknarg_20140212c_Exploration.log		Swampdog
Halgarad_20140213a_VulmanePathThroughHalgarad.log		Swampdog
Martok_Sea_n20_11.log		Theeper
Martok_Sea_n18_11.log		Theeper
Leth_Nurae_Log3.log		XionnaeVG
Xionnae_TombofLordTsang-1.log		XionnaeVG

These all had 0 NPCs total, meaning the bsList didn't find an SGONPCPawn entry. Note, they are also some of our original logs.

[Xen]

I just update UnrealBitLib with the latest vgr file counts. Try it out with some of those logs and let me know if that works. I know I was able to get DerogarsOutpost_Level1-5.log to parse after I made the changes, except it eventually ran out of memory.

[John Adams]

Yeah I found that compiling x64 solves that memory problem. Mostly a bandaid, of course. But works for me.

Edit: Nope, got no NPCs. Are you using VGVisualParser, or just running them through your own tool?