CareerCup

You can probably have the same application running on different hosts. When the number of clients host 1 is serving is more, then the traffic can be redirected to another host running the same application. The concept is very much load balancing.

Probably you can have the same application running on different hosts. When the number of clients,say, host 1 is serving is more, then the client requests can be redirected to another host running the application. This is a somewhat load balancing.

Probably you can have the same application running on different hosts. When the number of clients,say, host 1 is serving is more, then the client requests can be redirected to another host running the application. This is a somewhat load balancing.

> I have a single timer class which is running on a single port of a machine.

From this assuming that we can not use any load balancing mechanism like distributed on multiple host system. That means its running a single instance on a single port.

Implement a scheduler mechanism using loopback socket and epoll, when there is a request from client, insert that request as loopback socket to that scheduler which will check and notify the client when x is over.

Shared Memory concept....
Each time request comes.. start a new thread for that particular client.

Scale on same server
1. Use event driven model. There are dedicated threads to perform IO and CPU bound activities. This way, incoming network traffic wont get effected by timer object cpu activity.
2. Make use of event driven socket programming. Use artefacts like epoll, IOCP, kqueue.
3. By properly implementing the above (event driven, multithreaded, lock free DS and memory management), one can achieve 10k concurrent clients on a single commodity server.

Using more than one server
1. Have the client connect to a gateway
2. This can load balance between more than one server (LAN).
3. Gateway will thus manage server loads and can start up more servers in-case all servers are getting loaded. This can be achieved by notifying admin or loading dlls on server machines
We can have more than one gateway and a load balancer (hardware lvl) can be added before the gateway which load balances among the gateways. This will also make sure that gateway is not the single point of failure.

We can use N softirqs (N = no. Of cpu) to notify all the clients. These softirqs can retrieve their client list to be notified from percpu data structures to avoid any locking/contention. Why softirq? Because they can be scaled well, low latency n can run or scheduled in parallel on different cores at same time.
Thoughts?