Share Your Code

NoobHub

Posted by overtorment, Posted on August 8, 2012

GitHub URL: 
https://github.com/Overtorment/NoobHub

Opensource multiplayer and network messaging.
Connections are routed through socket server with minimum latency, ideal for action games.
Simple interface.
Server written on blazing fast nodejs. Socket connections, works great through any NAT (local area network), messages delivery is reliable and fast.

Repo includes server code and Corona lua client. You can test on my server, credentials are in the repo!

Lua code may serve as an example of how LuaSocket library works.


Replies

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Corona demo app included!

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Awesome it's great to have some multiplayer options other than pubnub. Thanks for this! Question, is there or will there be a way to set this up with our own server? If not is the server you are hosting this on free to use and reliable?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Server code included so you can use your own server easily.

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

Itching to try it out.

Any instructions on how to run this on our own server?

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

I have a query. When publishing the message

1
2
3
4
5
6
7
8
hub:publish({
          message = 
          {
          action  =  "ping",
          id=crypto.digest(crypto.md5,system.getTimer()..math.random()),
           timestamp = system.getTimer()
           }
});

you have not specified channel info.

So does that mean all messages will be sent to all devices regardless of what channel they have subscribed to? If that is the case, what is the point of having a channel in the subscribe function?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

@Satheesh, to run the server you need to instal node.js on the server, for example
sudo apt-get install nodejs
And the run it
node node.js
where "node .js" is my source for server part

Regarding the channel name, it's set when you are subscribing, and unchangeable. Thats the difference with pubnub.
To post and listen to another channel just create another instance via
 new_hub = noobhub.new();
and subscribe it to other channel.

Messages are posted only to the dedicated channel, not all channels

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

Ah excellent! Works like a charm!
Thanks a lot.

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Hey overtorment do you have any suggestions for cheap or free hosts that can host the node.js? Just found out my current hosting provider does not allow node.js installations atm due to security reasons :(

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Im using http://www.hetzner.de/en/ which is a leading european hosting provider. What you need from your hoster is the cheapest virtual server (VPS), so under the root you can install node.js

Until then just use my server for testing, it's details are already filled both in server and client code.

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

I found a very cheap VPS hoster in the U.S. http://www.lowendbox.com/ How much dedicated RAM would you suggest for a VPS using this? I know it depends on the amount of users, but assuming there will be a lot of people using it, most likely in the thousands and possibly at the same time, what do you suggest as a sufficient amount? Thanks!

Tyler

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Take a minimal at the moment. Once you'll feel the server cant take the load - change it to a more powerful one. There's no data to migrate or anything, so switching the server is as simple as just repointing DNS entry on the new server where you've set up a Node.js

If you want plain numbers...
Node spends a lil bit more than 10 kb of memory per connection.
I'm going to rewrite memory management a bit, and grant an additional dedicated buffer of 8-16 kb to each connection.
So you can count yourself, how much RAM you need.

As recently reported, some folks were able to make 1 dedicated server of node to handle 1 million connections simultaneously. That's a great result but I believe latency increases drastically. Plus, Node's garbage collector is not handling memory > 500 Mb good enough, and in case if it exceeds it, you'll need to do some rearrangements, or otherwise, again, responsiveness drops.

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Thanks for the advice! I rented a 256mb VPS and its running node.js amazingly! Thanks for this awesome library, I am currently developing 2 applications that are both using it! My question is, is there a way to terminate a noobhub channel subscription? IE say the user is playing a game and hes connected over hub = subscribe({.... then he quits the room and goes to the app's menu. atm I have a temporary method I've created in noobhub.lua that is as follows

function self:unsubscribe(params)
self.sock = nil;
end

Is this a good solution to removing a users connection to the server? The reason I am worried about this is because I don't want the user to back out of a game and then join a new one and then have 1 or more old open connections to the server which would take up unnecessary ram. With just a few users obviously an extra 40 or 50kb being used is no big deal, but when there are hundreds of users using it at the same time if they started accidentally recreating the connections that could become a big issue really quick.

Thanks again,
Tyler

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

@Richards
This is what i used for unsubscribing..

1
2
3
4
5
function self:unsubscribe(params)
        self.sock:close()
        self.sock = nil 
        self.buffer = ''
end

@overtorment Is that ok?

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Thanks Satheesh, that seems like a much better solution than what I came up with since youre actually closing the socket and not just only nilling it. Is this the best solution atm @overtorment?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Yeah, closing socket is the right thing, if you dont need it.
Could you guys please push this code to github? :-)

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

My game that uses Noobhub is close to being done, I'll be sure to share the store links once its out in case you want any testimonials/current examples for new/skeptical users to show how well noobhub works. My question is, whenever a player is dropped from connection in AutoLan, the host is notified, whenever someone leaves the channel in pubnub I believe they notify as well. Do you guys have any idea what the best way this can be done with noobhub? To notify players of a player's connection dropping or just a player quitting is through noobhub

Thanks,
Tyler

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Well this is how i solved this - every client pings channel every 5 seconds, and everybody listening have to reply "pong".
In that way i know who is still alive and can monitor network latency to use this data in lag compensation algorythm.
Pub/sub paradigm is famous with it's loose coupling of subscribers, so i would not want to change this in noobhub library.

Any testimonials would be really great! I would be really happy knowing that my code is used by people in projects.
Im thinking maybe i could write an article for Corona newsletter showing and explaining that corona is good for multiplayer games as well.

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Hey I'm running across a problem when publishing messages occasionally.

ERROR:

attempt to index field 'sock' (a nil value)

noobhub.lua:35: in function 'publish'

do you know why this error occurs randomly?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

looks like connection failure. need to add check for this and reconnection (or call Error callback). was planning to do this long time ago but still didnt manage...

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Another question I have, possibly related to this, if a server sends out messages in rapid succession, ie say the server sends a user a confirmation that theyre in the server, and then right after they send that they send a messae to everyone in the channel again but this time using a protocol that signifies to update the room player list, would the client be able to receive both messages at the same time? It seems like there's been some dropped messages when this has happened, usually this error comes up.

EDIT: Wow I feel really dumb lol, I just realized that ive been publishing to the old channel I had created and then unsubscribed to. Hopefully this fixes the problem for now :P Thanks!

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

if I got it correct - all messages will be delivered, and in correct order, as long as socket tcp/ip connection didn't break. rapid-ness or messages frequency do not matter.

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

@overtorment

Sometimes, the script stops ececuting midway. When I opened the nohup.out file, the following error was logged there.

1
2
3
4
5
6
events.js:66
        throw arguments[1]; // Unhandled 'error' event
                       ^
Error: write ECONNRESET
    at errnoException (net.js:768:11)
    at Object.afterWrite (net.js:592:19)

Any idea why that happens?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

looks like it tries to write to disconnected socket.
need to add a check for that

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

Will try catching the socket.write line do?
Something like

1
2
3
4
5
6
7
8
try
{
 sockets[socket.channel][c].write("__JSON__START__" + json + "__JSON__END__" /*+ '\r\n'*/);
}
catch(e)
{
console.log(e.message);
}

Or is there something else we have to take do?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

that might work, but node.js exception handling is a bit tricky so im not 100% sure.
maybe there's a solution in node.js socket documentation. like, a proper way to detect that socket is available for writing.

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

Ok. How about this?

1
2
3
4
5
process.on('uncaughtException', 
     function(err) 
     {
        console.log(err);
     });

I manually tried to access a closed socket, and the function caught the uncaught exception.

EDIT
Oops. Apparently, this is a very risky solution.
http://debuggable.com/posts/node-js-dealing-with-uncaught-exceptions:4c933d54-1428-443c-928d-4e1ecbdd56cb

I guess I'll just stick to the good old try..catch then

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

Ok try catching did not work. I still get errors.

However
process.on('uncaughtException',foo)
works very well.

What are your thoughts @overtorment?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

If it works - good job.
If you can - post the changes youll make Ill add them to repository.

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Hey, just thought i'd let you guys know my multiplayer game running via noobhub is very close to completion! I am close enough to start running testing sessions with other people! Hopefully starting 2moro I will be able to start testing it with others to make sure there aren't any unexpected bugs that I run across, and to get some feedback! Since your development of noobhub has made this game possible @overtorment, im extending an invitation to you and anyone else here interested in testing! Let me know if you're interested, shoot an email to richardsappstore@gmail.com and I'll send you the apk and let you know when i'll have people testing! If you're not interested or you're busy or something it's fine, just wanted to give thanks and a chance to see the tool in action!

Thanks,
Tyler

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

@overtorment

I see you have updated your node.js in github.
In the new version, sometimes I get the error

1
2
[Error: oob]
[Error: targetStart out of bounds]

This happens when I publish data continuously for a long time.
After this error occurs, the socket which published all the data, cannot publish any more data.. It receives data though..

Ths answer at stackoverflow says this can happen when you try to manipulate the length of a fixed size allocation in memory by the += method

Any ideas, overtorment?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Must be my new mechanism of data storage on Buffer memory...
Looks like buffer size overflow. Need to debug this.

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

Our multiplayer game DRAW N GUESS which uses Noobhub is alive in the market..
(actually the game was released some months back. But multiplayer games never worked properly, hence the poor reviews. The latest version which was released recently uses NoobHub and it kicks ass)

Play Store : https://play.google.com/store/apps/details?id=com.timeplusq.drawnguess

App Store : http://itunes.apple.com/app/draw-n-guess-multiplayer-online/id532137502?mt=8

I was planning on crediting you @overtorment, but unfortunately forgot about it in the chaos before app release :D Will do so in the next update.
So, how do you want to be credited?

P.S I'm still using the old version of the node-js code. As I mentioned earlier, the latest version encounters buffer errors, so once I get the time to fix that, I would be updating to the new version! Cheers!

Jack01
User offline. Last seen 10 weeks 6 days ago. Offline
Joined: 11 May 2012

Hey thx a lot for this code!! I dove into node.js and it looks real cool. Could you give some tips?

Did you implement matchmaking? Suppose I want to find another player to play a 1 vs 1 game.

I suppose automatic a channel needs to be created and later removed for each pair of players?

Would you advise to use a framework like Express of Meteor to build further on your code and implement these feautures?

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Hey Satheesh, is your game realtime? If so, do you have an effective way of checking if the users in game are still in game? I currently use this function:

local checkStatus = function(event)
if player2 ~= "" then
if player2temp == 2 then
player2temp = 1
print ("Player 2 Is Warned")
elseif player2temp == 1 then
print ("PLAYER2 LAST CHANCE!!! CONFIRM YOURE HERE BRO!!!")
hosthub:publish({
message = {
action = "ping",
id = idnumber,
protocol = "-1",
pos = 2
}
})
dropTimer2 = timer.performWithDelay(1000, function()
-------------------------------------------------------------------------------------------------
--IF THE HOST DOES NOT RECEIVE A PONG Protocol: -1 FROM THE
--CLIENT WHOSE POSITION = 2 WITHIN 1000ms, then this function is
--ran, however if it does receive then dropTimer2 is cancelled.
-------------------------------------------------------------------------------------------------
print ("Player 2 Is Gone")
numClients = numClients - 1
dropPlayer({pos = 2})
end, 1)
end
end
if player3 ~= "" then
if player3temp == 2 then
player3temp = 1
print ("Player 3 Is Warned")
elseif player3temp == 1 then
print ("PLAYER3 LAST CHANCE!!! CONFIRM YOURE HERE BRO!!!")
hosthub:publish({
message = {
action = "ping",
id = idnumber,
protocol = "-1",
pos = 3
}
})
dropTimer3 = timer.performWithDelay(1000, function()
-------------------------------------------------------------------------------------------------
--IF THE HOST DOES NOT RECEIVE A PONG Protocol: -1 FROM THE
--CLIENT WHOSE POSITION = 3 WITHIN 1000ms, then this function is
--ran, however if it does receive then dropTimer3 is cancelled.
-------------------------------------------------------------------------------------------------
print ("Player 3 Is Gone")
dropPlayer({pos = 3})
numClients = numClients - 1
end, 1)
end
end
if player4 ~= "" then
if player4temp == 2 then
player4temp = 1
print ("Player 4 Is Warned")
elseif player4temp == 1 then
print ("PLAYER4 LAST CHANCE!!! CONFIRM YOURE HERE BRO!!!")
hosthub:publish({
message = {
action = "ping",
id = idnumber,
protocol = "-1",
pos = 4
}
})
dropTimer4 = timer.performWithDelay(1000, function()
-------------------------------------------------------------------------------------------------
--IF THE HOST DOES NOT RECEIVE A PONG Protocol: -1 FROM THE
--CLIENT WHOSE POSITION = 4 WITHIN 1000ms, then this function is
--ran, however if it does receive then dropTimer4 is cancelled.
-------------------------------------------------------------------------------------------------
print ("Player 4 Is Gone")
dropPlayer({pos = 4})
numClients = numClients - 1
end, 1)
end
end

And then here is the actual pinging loop that the host runs:

local checkping = timer.performWithDelay(2000, function()
hosthub:publish({
message = {
action = "ping",
id = idnumber,
protocol = "0",
player1 = player1,
player2 = player2,
player3 = player3,
player4 = player4
}
})
end, -1)

With this system, the connection people have with the host often gets sketchy, and they end up disconnecting even though they still have running internet. I was planning to release my game last night then in my final wave of group testing noticed this connection dropping problem. I'm not sure if every message is supposed to be received or if it is common for any packets to be lost, but it seems to be happening with my game. Anyone have any suggestions?

Thanks,
Tyler

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

Hey @overtorment, I am still having issues with clients and servers losing connection with each other frequently with the functions that I posted previously above this post. What's frustrating is that I am sure the client isn't actually dropping connection, it's just not getting or sending a message or two, which causes the host to think they're gone and then drop them, or vice versa where the host doesn't send out a message or two or the client doesn't receive them so the client thinks the host is gone and they return to menu. I reread what you had said originally about the best solution to check if clients still exist is just to have them ping and pong each other. Then I took a look at the node.js code and I noticed that when a socket connection closes, a console.log is set to show the client id and channel that it disconnected from. Is it possible to send this to the clients noobhub connection? My game Math Race Online is now live on the NOOK Store for free until I can get this connection problem sorted out. Any help would be greatly appreciated.

Thanks,
Tyler

Satheesh
User offline. Last seen 34 min 8 sec ago. Offline
Joined: 25 May 2011

@Richards

Yes our game is Real Time. But I have never gotten connection issues like you mentioned.

In fact, I never bother pinging users. When a user leaves a game (voluntarily or if he gets a phone call or something), I just publish a "i am leaving" message.

The obvious downside of this method is it cannot detect if the user's internet connection stops midway. So I will be implementing the "ping pong" method soon. Apart from that, the game is working pretty well as it is.

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

@Satheesh

I "herpderped" pretty hard up until this point and never thought about just sending out a pong that signals to remove a user when they back out/close the app/device sleep but you still have the problem you pointed out because if the users internet goes out then the host won't know. I also realized by checking node.js logs that when the user unsubscribes the server could notify the client, however the only way this could be an improvement over just sending out a ping before unsubscribing is if node.js is able to know when the user loses internet connection and then sending out a message to clients, but I'm not sure if node can detect that or not.

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Hello guys, I'm finally back from vacation.
Congratz on your released apps, I'll try to check them out nearest future.
Regarding the problems with server code - me and my co-author will look into it. BTW, It would be best to report issues directly on github https://github.com/Overtorment/NoobHub/issues

Regarding notification about client's disconnection - I dont plan to implement this, as beauty of publish/subscribe paradigm is in it's simplicity.
For my own game on Noobhub I've implemented ping-pong algorithm which Im happy about, as it allows me to measure message delivery latency as well to use in lag-compensation.

jonjonsson
User offline. Last seen 15 min 23 sec ago. Offline
Joined: 30 May 2011

Hi,

thanks for this beautiful solution.

I'm storing incoming messages for users that are not online yet in database. When they come online I send them their stored messages.

I'm wondering, when sending multiple messages from server to client, is there a danger of flooding the client? Or is this just a steady stream? Is it better to put few milliseconds between messages?

dgaedcke
User offline. Last seen 9 hours 57 sec ago. Offline
Joined: 17 Apr 2010

Have you guys seen this project??
http://luvit.io/
It's a node.js port done in Lua....enjoy

qaisjp
User offline. Last seen 13 weeks 6 days ago. Offline
Joined: 15 Jul 2011

AppFog is a really good cloud service for web apps.

qaisjp
User offline. Last seen 13 weeks 6 days ago. Offline
Joined: 15 Jul 2011

HI, thanks for this really cool solution.
My game is supporting multiplayer and will support 1 on 1 racing (later i'll see if i can add support for even more), how can I find matches to play with? That is, you cannot play until you have a partner.

For example:
player1 pressed Multiplayer and then the game "Finds matches" (connects to the server)
player1 has to wait until another player wants to play multiplayer.
player1337 pressed Multiplayer and then the game "Finds matches"
player1337 and player1 has been matched and have been given their own channel.
player1337 and player1 tell their partner some information (username, clothing etc)
The multiplayer match between player1337 and player1 starts.
player1337 and player1 constantly sync information, such as positions.

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Hello, thanks for checking out this library.

This lib allows only basic communication between clients, eg. send/receive json messages.
All game logic should be implemented by you.

Minion Multimiedia
User offline. Last seen 7 weeks 14 hours ago. Offline
Joined: 19 Oct 2010

How solid is the server in real world use?

Richards App Store
User offline. Last seen 2 days 3 hours ago. Offline
Joined: 15 Dec 2011

@minion

I've been using it to host my game Math Race Online across nook, kindle and android for 4-5 months using a cheap like $2 a month VPS, and it's worked solidly. How solid it performs also depends on how well you program your game obviously, but the library itself works great when used live.

Minion Multimiedia
User offline. Last seen 7 weeks 14 hours ago. Offline
Joined: 19 Oct 2010

@Richards
Thank you for the info. I am testing things on a virtual box (running Ubuntu) on my Mac right now... The server .js crashes somewhat frequently and then... Just wanted to make sure that running it on an computer works before start looking into VPS/building my own...

@overtorment
I added some statements that return a true/false (if a connection is dropped/or can't be created). Is it okay to push these changes to GitHub?

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

Yeah, why not, Ill review the changes.

Also, crashes might be because of latest node.js.
As another reason - misuse os server (or poorly coded client), I didnt implement any buffer overflow checks... Gona add them sometime.

Jack01
User offline. Last seen 10 weeks 6 days ago. Offline
Joined: 11 May 2012

I managed to use noobhub for creating user accounts and logging in with Corona with the nice node.js / MongoDB account create / login system on https://github.com/braitsch/node-login

It works nice now, sending back messages like username allready exists etc. Passwords are encrypted in Corona with crypto.sha256 and a salt before sending it to the server, which should make it a secure system.

I am using at the moment "forever" to keep node.js running. I have a gameserver.js which takes care of accountmanagement and it should send game info between devices. A webserver which runs the node-login system, maybe later I make it a support site for the game. And a node.js proxyserver to serve the url for the website. All is running on Hetzner.de on a Vserver 12 with a minimal Ubuntu OS, without Apache :D Hope 1 GB of memory will do at start. Excellent tip Hetzner, thx! They are fast, cheap and a very professional host. I had some bad experiences with another vps host.

Now I am ready to connect 2 iPhones.

Any chance on someone sketching me out an example to put 2 players in 1 channel? I can imagine u dont wanna share the code but some pinpointing would help me greatly. Thx.

overtorment
User offline. Last seen 2 weeks 4 days ago. Offline
Joined: 31 Aug 2011

@Jack01
thanks for using this lib!
Here's general solution for your question, which is usually called 'match-making'

All users of the game should be subscribed to the same channel, say,
"my-super-game-lobby". This is called "lobby".
Once somebody needs to find a gaming partner, he posts a message to the lobby channel, like, 'who_wants_to_play'.
Every client on lobby who is up for a game, answers 'i_wanna_play'.
Ofcourse, initial 'who_wants_to_play' message should carry unique identifier of request, and 'i_wanna_play' answers should carry this identifier as well (and unique user id also).
Once game starter got responses to his 'who_wants_to_play' request, he can choose (random?) user to play with, so he posts a message to lobby 'lets_play_in_channel', carrying id of the user to play with, and new unique name of the channel. The user who this message was intended for should instantly subscribe to this new channel, where all in-game interactions will happen. That's it, game begins.

This whole process is called 'negotiation', and ofcourse it can be more complex than the brain-dead-simple variant I described :-P

Jack01
User offline. Last seen 10 weeks 6 days ago. Offline
Joined: 11 May 2012

Thx, that doesnt seem to hard to implement. I guess the subscribing to the new channel should be done in Corona.