Page 1 of 1

Dev Chat: 03/31/2014 - Unreal Channels

Posted: Tue Apr 01, 2014 1:01 pm
by John Adams
This conversation between Xen, Scatman, theFoof and Jabantiz is related to how we plan to implement the Unreal Engine "channels" for spawn (and other) data.

[quote][18:37] <@Xen> So Scat, how do I get the current character id for the client?
[18:37] <@Xen> I was going to try and load up the appearance in game
[18:40] <Scatman> i don't think i got there. I started a 'Character' class
[18:40] <Scatman> the Client object will hold a pointer to one
[18:40] <Scatman> and when we get the char ID then we can allocate the object and load it from the database
[18:40] <@Xen> Okay, I didn't think I saw a connection to it yet
[18:40] <Scatman> yah it's not there yet
[18:41] <Scatman> gonna fix this cp bugs first
[18:41] <@Xen> Now, as far as the unreal channels that hold npcs, pc's, objects in the worlds, etc: Does it make sense to have that tied to each client as well
[18:41] <Scatman> well, right now
[18:41] <@Xen> An array ao channels so to speak
[18:42] <@Xen> of*
[18:42] <Scatman> yes. but are those channels unique to the client?
[18:42] <Scatman> or to the zone?
[18:42] <@Xen> client
[18:42] <@Xen> That's another thing, I believe in VG you can see mobs across zone lines
[18:43] <Scatman> then yeah, each client should probably hold an array
[18:43] <Scatman> a statically allocated array
[18:43] <Scatman> so we're not doing allocations all the time
[18:43] <Scatman> oh yeah?
[18:43] <Scatman> then in the case maybe we have a global Channel Manager
[18:43] <Scatman> a map<character id, ChanneList>
[18:43] <Scatman> so all zones can see it
[18:43] <@Xen> k
[18:44] <Scatman> I'd make a Channel class
[18:44] <Scatman> and a ChannelList class
[18:44] <Scatman> maybe call it UnrealChannel
[18:44] <Scatman> because we need ChatChannels too
[18:44] <~john> yeah did I forget to tell you that mobs can chase your ass across the zone and kill you? No safety in VG!
[18:44] <Scatman> heh nice
[18:44] <@Xen> in C# to make a list of an object I declare it as List<UnrealChannel>. What do I use in C++?
[18:45] <@Xen> Right john, but I believe when you cross the zone line there is that little pause as it reloads all the channels. So that mob chasing you might have been on channel 25 in the previous zone but is now in channel 10.
[18:45] <@theFoof> http://en.cppreference.com/w/cpp/container/vector
[18:45] <@theFoof> http://en.cppreference.com/w/cpp/container/list
[18:45] <@Xen> Thanks Foof
[18:46] <Scatman> well Xen, it'd be similiar to a Dictionary<>
[18:46] <@theFoof> and map is like dictionary.
[18:46] <@Xen> I just have to make sure it is standard C/C++ right
[18:46] <Scatman> yeah. std::map
[18:46] <@Xen> k
[18:46] <Scatman> you can do 'using namespace std;' at the top
[18:46] <Scatman> so you don't have to include std all over
[18:46] <Scatman> and #include <map>
[18:46] <Scatman> check ClientList for an example or something
[18:46] <@Xen> k
[18:47] <@Xen> thanks
[18:47] <Scatman> but your unique key will be the character id
[18:47] <@theFoof> http://en.cppreference.com/w/cpp/container/map
[18:47] <@theFoof> might as well give you the map link heh
[18:47] <@Xen> heh
[18:47] <Scatman> map<uint32_t, UnrealChannel *>
[18:48] <@Xen> Is uint32_t short for unsigned long?
[18:48] <@theFoof> unsigned int
[18:48] <Scatman> it's guarnanteed to be a 32 bit unsigned integer
[18:48] <Scatman> guaranteed
[18:48] <@Xen> k
[18:49] <@Xen> wait I though long was 32 bit int as well
[18:49] <@Xen> or is that system specific
[18:49] <Scatman> depends on your architecture :P
[18:49] <@Xen> gotcha
[18:49] <@Xen> So I don't need to make a UnrealChannelList class?
[18:50] <Scatman> i would. to encapsulate your map<>
[18:50] <Scatman> then you can write functions like AddChannelsForClient(), GetChannelIndexForClient(), etc
[18:50] <@Xen> It would be a list/vector, not a map right?
[18:50] <Scatman> bunch of setters and getters
[18:50] <@Xen> map is for the master list
[18:51] <@Xen> Each client has a list of Channels
[18:51] <Scatman> Each client has a list of channels, correct
[18:51] <@Xen> public class UnrealChannelList:vector<UnrealChanel>
[18:51] <Scatman> nope
[18:51] <Scatman> Check out packetStructList.h
[18:52] <@Xen> map<uint32_t, UnrealChannelList> global_chan_list
[18:52] <@Xen> k

[18:54] <Scatman> class UnrealChannelList {
[18:54] <Scatman> public:
[18:54] <Scatman> private:
[18:54] <Scatman> map<uint32_t, UnrealChannel *> channels;
[18:54] <Scatman> map<uint32_t, vector<UnrealChannel *> > channels
[18:54] <Scatman> };

[18:54] <Scatman> i'd use one of those 2 map structures
[18:54] <Scatman> the first one would be more C-like
[18:54] <Scatman> with a C array
[18:54] <Scatman> or you can allocate a vector of channels for each character id
[18:55] <@Xen> Okay, I see
[18:56] <@Xen> And somewhere we'd declare a global channel list of type UnrealChannelList to handle all the characters?
[18:57] <Scatman> yes
[18:57] <Scatman> look at Main.cpp
[18:57] <Scatman> at the top
[18:57] <Scatman> you'll see all similar declarations
[18:57] <@Xen> k
[18:57] <Scatman> or those global lists
[18:57] <@Xen> There's a max of 1024 channels per character
[18:57] <Scatman> oh that many huh?
[18:57] <@Xen> yeah
[18:58] <@Xen> But I believe many of the channels will be tied to the same npcs between characters
[18:58] <Scatman> ok then i'd allocate and dellcate them as we go then
[18:58] <Scatman> that's what the create/destroy flags are for anyway right?
[18:59] <@Xen> Probably but I thought I saw in the unreal code that they allocated all 1024 slots at the beginning. I may be wrong on that though
[18:59] <Scatman> yeah. it'd be faster that way
[19:00] <Scatman> ah screw it. let's allocate them all in the beginning
[19:00] <Scatman> memory is cheap!
[19:00] <@Xen> lol
[19:02] <@Xen> Like I said, these channels will be point to the same data at times. If we are both in the same area than a lot of our channels will be pointing to the same npcs, objects, etc
[19:05] <@Xen> So I'm guessing we will need a master list holding all the currently spawned npcs as well
[19:06] <~john> MUTEX!!
[19:06] <@theFoof> so is the chunk server literally going to be per chunk or like if you're in qalia the chunkserver covers all chunks in qalia?
[19:06] <Scatman> yes xen
[19:06] <Scatman> and the channels will simply hold pointers
[19:06] <Scatman> to those objects
[19:07] <Scatman> 4/8 bytes!
[19:08] <@Xen> That's a good point Foof. What is the job of the chunk server
[19:08] <Scatman> i was thinking each "Chunk Server" would hold it's own npcs and such
[19:09] <@Xen> Ahh so it would handle the logic for how those npc's move, etc
[19:10] <@theFoof> i get that but on vanguard it's not like in eq2 where the zones are all separate, a chunk is just a gridded area on a map
[19:10] <Scatman> yeah i didn't really know that
[19:10] <Scatman> is there anything unique to a chunk?
[19:10] <Scatman> map?
[19:11] <@theFoof> well let's say you change chunks, there is a loading process, but you can see the other side of the chunk zone line without changing
[19:11] <@theFoof> spawns can chase you across ect
[19:11] <@Xen> Yeah
[19:11] <Scatman> ok
[19:11] <Scatman> let's keep it like this. it wouldn't be hard to handle the NPCs over to other chunks
[19:11] <Scatman> because eventuall we want to have the ability to load Chunks on different servers
[19:12] <Scatman> you know, when we become famous
[19:12] <@Xen> lol
[19:12] <Scatman> ;)

[19:41] <@Xinux> when i was messing around last night
[19:42] <@Xinux> http://toxic04.com/1.JPG
[19:42] <@Xinux> http://toxic04.com/12.JPG
[19:43] <Scatman> heh cool
[19:43] <~john> I need to block pasting so Xinux has to post his cool screen shots on the forum.
[19:43] <@Xinux> lol

[19:44] <@Xen> Can we use C++11 standards, or just stick to C++98/03?
[19:44] <Scatman> i think we're ok with C++11
[19:45] <Scatman> i believe gcc has more implemented than microsoft
[19:45] <Scatman> if we run into trouble we'll fix it
[19:53] <@Xen> was debating using std::array instead of std::vector for the channels property inside UnrealChannelList
[19:54] <Scatman> i believe vector<> is the prefered list structure
[19:54] <Scatman> i think bjarne himself said that
[19:56] <@Xen> Looks like vector<> gives you more functionality for deleting/inserting, etc
[19:56] <@theFoof> they're also different, a vector has dynamic size where an array is fixed i believe
[19:56] <@Xen> I just though array<> because we know the size is static
[19:56] <@theFoof> std::array rather, obviously arrays are fixed but I haven't used a std::array
[19:56] <@Xen> But it looks like I can set capacity on vector<> so it shouldn't have to reallocate
[19:57] <Scatman> yeah might be better to use an array since the size is static
[19:57] <Scatman> vectors are good at scaling which we don't need to do really
[20:00] <@theFoof> being that we're using a fixed size for the variable you could just use a c style array though, whatever you want to do

[20:07] <@Xen> std:array is supposed to give some extra functionality and it said something about hiding a pointer
[20:09] <@Xen> I can always switch later
[20:09] <Scatman> yah its not a big deal
[20:10] <@Xen> Scat, are we basically doing 1 class per file, or can I combine UnrealChannel and UnrealChannelList into one file?
[20:10] <Scatman> you can do them in the same
[20:10] <@Xen> k

[20:22] <@Xen> Short article about std:array vs built-in C/C++ arrays: http://www.devx.com/cplus/Article/42114
[20:44] <@theFoof> keep in mind with that global channel list that you probably want to use a mutex, I'm assuming multiple threads can access it
[20:45] <@Xen> I'll never get this thing written :)
[20:45] <@Xen> But I'm learning a lot so that is good
[20:46] <@theFoof> heh will be minimally more code
[20:50] <@Scatman> yep :)
[20:51] <@Xen> You guys okay with me typdefing this: typedef map<uint32_t, array<UnrealChannel, MAX_CHANNELS>> ChannelMap;
[20:52] <@Xen> Or should I just be explicit
[20:54] <@Scatman> you can typedef it if you want i guess. for what, the iterators?
[20:54] <@Scatman> you'll only have one right?
[20:54] <@Xen> Yeah
[20:54] <@Xen> I guess it's better to just be explicit in this case
[20:55] <@Xen> I keep getting hung up thinking I need to do a new on every object like I do in C#
[20:55] <@Scatman> heh nope!
[20:55] <@Xen> So I thought my new statement was going to look like:
[20:56] <@Xen> channels = map<uint32_t, array<UnrealChannel, MAX_CHANNELS>>();
[20:56] <@Xen> But then I realized I didn't need to do new
[20:56] <@Scatman> nope
[20:56] <@Scatman> just statically allocate it
[20:56] <@Scatman> much easier
[20:57] <@Xen> Well I can't statically allocate the map since I have no idea how many characters will be on
[20:57] <@Xen> right?
[20:57] <@Scatman> you can allocate it if you want
[20:57] <@Scatman> but there's no reason to
[20:57] <@Scatman> the class would store 4 or 8 bytes on the stack. and the rest of the implementation would be in the heap
[20:57] <@Xen> Sorry I know these are dumb questions
[20:57] <@Scatman> not dumb!
[20:58] <@Xen> When I add a new item to the map I will be statically allocating the channel array<> though
[20:58] <@Xen> since I know it will be 1024 size
[21:00] <@Scatman> i've always dynamically allocate the map's key/values unless they are primitive types. i'm not sure how the map handles statically allocated things...
[21:00] <@Scatman> i'll look it up
[21:03] <@Scatman> looks like if you store it statically you'll need to implement a deep copy constructor
[21:05] <@Scatman> it'll have to copy the objects. so copying pointers (aka integers) is way more efficient than copying objects
[21:10] <@Xen> By dynamically allocate, do you mean: map<uint32_t, array<UnrealChannel, MAX_CHANNELS>>* channels; ?
[21:12] <@Scatman> no. the map itself will stay statically allocated
[21:12] <@Scatman> you would dynamically allocate the array<>
[21:13] <@Scatman> map<uint32_t, array<UnrealChannel, MAX_CHANNELS>*> channels;
[21:13] <@Scatman> so just move the asterix back one :)
[21:13] <@Xen> sorry C# just spoils me :P
[21:14] <@Scatman> heh
[21:15] <@Xen> in C# this would be Dictionary<uint, List<UnrealChannel>> channels = new Dictionary<uint, List<UnrealChannel>>();
[21:16] <@Xen> For some reason that is more clear to me heh
[21:16] <@Xen> It's those damn pointers that are throwing me off
[21:17] <@Scatman> right and each List<UnrealChannel> would also need to be allocated with new, right?
[21:18] <@Xen> yeah to add an item I would do channels.Add(new UnrealChannel() { properties to be set here});
[21:19] <@Scatman> ok. so same thing
[21:19] <@Scatman> but you don't use 'new' for the dictionary (map) itself
[21:20] <@Xen> Okay, so I'm going to typedef the array because I'm using it in a couple places:
[21:20] <@Xen> typedef array<UnrealChannel, MAX_CHANNELS> UnrealChannelArray;
[21:20] <@Xen> Then I declared the map like this:
[21:20] <@Xen> map<uint32_t, UnrealChannelArray*> channels;
[21:21] <@Xen> Then in my function where I'm adding a new element to the map I did this:
[21:21] <@Xen> UnrealChannelArray* character_channels = new UnrealChannelArray();
[21:21] <@Xen> character_channels->fill(NULL);
[21:21] <@Xen> channels[character_id] = character_channels;
[21:21] <@Xen> Am I doing that completely wrong?
[21:22] <@Scatman> no that looks fine
[21:22] <@Scatman> i'm just not sure why fill(NULL) does
[21:22] <@Scatman> sets all the objects to NULL?
[21:22] <@Xen> It sets all the elements in the array to that value
[21:22] <@Xen> yep
[21:22] <@Scatman> k
[21:22] <@Scatman> that looks good
[21:22] <@theFoof> I don't think you want to use fill if you're not using an UnrealChannel pointer in the UnrealChannelArray def
[21:23] <@theFoof> looks like you have the actual object
[21:23] <@Scatman> oh he's not, you're right
[21:23] <@theFoof> but again I haven't used std::arrays before
[21:23] <@Xen> Obviously I should probably check that the character isn't already in the map, although I guess it would just overwrite it
[21:23] <@Scatman> it would overwrite it, but if you didn't free the memory you'll cause memory leaks
[21:23] <@Xen> k, I'll put a check in
[21:24] <@Xen> So, I don't need to fill at all? I'm trying to think why I was doing that
[21:25] <@Scatman> yeah foof is right
[21:25] <@Xen> I think I was gong to check if the channel was NULL before messing with it but I don't think I need to
[21:25] <@Scatman> you're storing the actual objects in that array
[21:25] <@Scatman> no need to fill(). you're UnrealChannel constructor will take care of any initializations
[21:25] <@Xen> gotcha
[21:25] <@Scatman> you'll need a boolean in the UnrealChannel class to see if it's in use or something
[21:27] <@Xen> Yeah I'm thinking I might not need that. When the channel closed, I will just call my CloseChannel function which does some resetting of channel data (setting seq_num = 0,etc)
[21:28] <@Scatman> kewl
[21:28] <@Scatman> there ya go
[21:28] <@Xen> You guys rock
[21:30] <@Scatman> glad to help

[22:22] <@Xen> scat, do you have time to do a quick code review before I commit this?
[22:22] <@Scatman> sure
[22:22] <@Scatman> you can just commit it
[22:22] <@Scatman> i'll grab it and look it over
[22:23] <@Scatman> unless you want no evidence on SVN :D
[22:23] <@Xen> lol
[22:26] <@Xen> okay it is commited. I haven't added the global list yet and I'll still need to add a few more properties to UnrealChannel.
[22:26] <@Scatman> ok

[22:32] <@Scatman> xen, everything looks good. You'll need to use the Mutex class in common/ to make sure the map<> is protected from multiple threads
[22:32] <@Scatman> but other than that it's a great start
[22:33] <@Xen> cool thanks
[22:34] <@Xen> I'll probably start on the unreal actor classes next because the channels are going to be pointing to them
[22:34] <@Scatman> i'm going to re-commit. added to the makefile and i have some changes too (not in your code)
[22:34] <@Xen> You can take out those semi-colons if you want
[22:35] <@Scatman> k
[22:35] <@Xen> Okay I better get some sleep. Looks like I am staying home with the sick wife and kid tomorrow ;P
[22:35] <@Scatman> heh goodluck
[22:36] <@Scatman> and good night
[22:36] <@Xen> night guys
[22:36] <@Xen> and thanks again for all the C++ help
[22:36] <@Scatman> np[/quote]