Quick Guide: Defining PingFederate Subcluster Servers [Updated - 11.29.15]
When setting up PingFederate in a highly-available architecture with a large number of nodes, it can be beneficial to add dedicated sub-cluster State Servers to manage session information across the cluster. A State Server is essentially a PF runtime node that does not handle normal SSO traffic, instead, it is dedicated to managing and centralizing specific federation session information for your deployment. While It’s not difficult to configure a State Server, you do need to have your logical infrastructure in mind to keep things straight. For example, let’s say you 4 runtimes across 2 data centers. Instead of having all 4 runtimes share session data directly, you'll want to setup a single State Server in each data center to minimize traffic that needs to traverse the data center connection. So in your first data center have 2 runtimes (rt1 and rt2) and you want to setup 1 state server (ss1). In your second data center, you'll have a mirror of the setup (rt3, rt4, ss2). The goal is to get rt1 and rt2 to direct session info/requests to the local state-server (ss1). Same for rt3 & rt4 w.r.t. ss2. Here's a simple diagram that shows the layout:
To set this up, do the following:
On <rt1>, you would edit your run.properties and set the cluster info as you normally do -
pf.cluster.node.index=1 pf.cluster.auth.pwd=<pwd> pf.cluster.encrypt=false pf.cluster.bind.address=<bind IP> pf.cluster.bind.port=7600 pf.cluster.failure.detection.bind.port=7700
On <rt2>, you would do the same thing:
pf.cluster.node.index=2 pf.cluster.auth.pwd=<pwd> pf.cluster.encrypt=false pf.cluster.bind.address=<bind IP> pf.cluster.bind.port=7600 pf.cluster.failure.detection.bind.port=7700
On <ss1>, you would do the same thing
pf.cluster.node.index=3 pf.cluster.auth.pwd=<pwd> pf.cluster.encrypt=false pf.cluster.bind.address=<bind IP> pf.cluster.bind.port=7600 pf.cluster.failure.detection.bind.port=7700
Notice that each runtime has a unique pf.cluster.node.index. In your 2nd data center, you have a similar setup except that your runtimes would be pf...index=4, pf...index=5 and state server pf...index=6.
Up until now, you've followed the steps to configure a normal cluster where all 4 runtimes (and 2 state servers) are sharing all session data equally. Now, in order to direct your runtimes to share with a specific subset of servers, you'll need to modify the following file on each server in your cluster:
For example, on <rt1>, you will need to set:
This means that <rt1> will share state information with itself, <ss1> and <ss2>. If this field is left blank, all nodes will share state info with each other. Also, if you did not have the the local runtime index listed, then the runtime would have no state information stored locally and every state request would need to be sent to <ss1> and <ss2>.
On <rt2>, you will modify the file the same way and set:
This instructs <rt2> to share with <ss3> and <ss6>. As you modify the files for <rt3>, <rt4> and <ss2> you end up with a information flow that looks like this:
Why specify certain node indexes?
When PF requests session information from the cluster, it automatically sends out a request to ALL the nodes listed in the preferred.node.indicies and will wait until it hears back from a majority of the members of the group. So if there are 3 servers listed, then the runtime will wait until it gets agreement from a majority of the nodes (2 of 3 in our case). So if a request is sent to <rt1>, there is a very highly likelihood that <rt1> and <ss1> will respond before the <ss2> and move on.
The final step required is to ensure that your load-balancer directing traffic to the PingFederate runtimes is persisting user sessions to one data center or the other. This allows PingFederate to more efficiently manage sessions and minimizes the amount of cross-datacenter traffic required.
What about OAuth 2.0 or OpenID Connect?
Unfortunately, OAuth and OIDC tokens do not have the ability to utilize the State Server cluster capabilities inside of PingFederate (as of 8.0.3). In cases where communication between runtime nodes across data centers is not possible, a self-contained token (JWT) may be an option.
---------------------------- Ian Barnett Ping Identity Practice Director