Configuring Safewhere*Identify for high-load environment

Overview

We have spent a fair amount of time to improve performance for Safewhere*Identify 5.0 and the result is a product with a great numbers of configurable settings that you can use to fine tune an Identify installation to achieve highest loads.

Tuning Identify installation

We are going to go through a bunch of settings that may affect Identify operation as well as performance. The key idea is that the less Identify needs to access database the higher performance result it can achieve.

Session state

We decided to drop support for SQL SessionState because it doesn’t scale well. Alternate options are described in http://docs.safewhere.com/session-state-support/.

Understanding common system log target

Previous versions of Identify logs information to:

  • IdentifyAudit SQL database: data such as token issuances, user requests, changes made to Identify database. (1)
  • Windows Event Log: errors, some important information events, and also user requests (2)
  • And flat file is used for diagnostic logs (3)

From Identify 5.0, we have reworked the all the logging thingies:

  • IdentifyAudit SQL: this is still working as-is with a new option to disable logging to it.
  • Introduce the system log target concept: a single log target that is used for both (2) and (3). The setting is placed at System Setup -> Logging -> Log Target. Currently it supports logging to flat file and Windows Event log.

Audit log

By default, Identify logs token issuances as well as changes made to its database to the IdentifyAudit database. A disadvantage of logging to SQL database is that it slows down token issuances significantly. The impact is even bigger when the SQL database is set up with mirroring.

An alternate option that yields better performance is to log those pieces of information to common log target. This can be done by tweaking two settings found in the System Setup -> Logging page:

  1. Enable audit logging: make sure it is unchecked.
  2. Enable revision logging: make sure it is checked.

Log target

So far, our benchmark hasn’t shown any clear difference in load result between flat file and Windows event log.

Audit user requests

This is about the legacy user requests which are described at http://docs.safewhere.com/audituserrequest-2/. Recommended configuration for best performance is:

  • Log user requests to database store: must be unchecked.
  • Log user requests to Windows Event log: you can check this setting if you want to log all requests to Windows Security Event log.
  • Log user requests to the "Log target" as specified in the setting below: you can check this setting if you want to log all requests to the common system log target.
  • Or if you don’t need this legacy user requests log, you can simply uncheck all three settings.

Store login information in cookies

Identify needs to store information about all the Service Providers and Identity Providers who participate on a login session so that it can log them out at SLO time. By default, Identify stores the participants in the IdentifySession database. To avoid using SQL database for this purpose, you can instruct Identify to use browser’s cookies instead. The setting can be found at System Setup -> Store login participants in cookies. Please note that this option can’t be used together for SOAP logout.

Store Saml2 incoming assertions in memory

Identify stores IDs of SAML 2.0 incoming assertions from upstream IdPs in SQL database to use later on for replay attack detection. This usually generates a lot of queries to SQL database which may be a performance hit. For deployments that is single-server setup or uses sticky load balancing, switching to store the IDs in memory can help. You can find the setting at System Setup -> Store Saml2 incoming assertions in memory.

Delay loading fresh data

As-is, Identify caches almost all data in memory for fast process. In order to deal with stale data, it needs to perform a single read to a special flag in database for every request to find out if it needs to invalidate and refresh its cache. For installations that need to handle high load but don’t need truly real time data, namely it is acceptable that an updated connection only takes effect after 1 minute or 2, you can configure Identify to check database for new data periodically instead of for every request. You can do this by going to System setup -> SQL Cache duration -> enter a positive value for it. Recommended value is from 60 seconds to 300 seconds.

User cache

Caching user is a little bit trickier than caching other types of data because users are potentially updated a lot more frequently. Since there is no one-size-fit-all solution for this, we decided to provide a few options so that you can pick one that is best suit to your specific setup.

  • Setting: System setup -> User Cache setting
  • PerRequest: For every web request to Identify that needs a local user, the user is loaded into memory and reused as long as the request lasts. This is the default mode which is used by previous versions too. When you aren’t sure which one to use or your installation updates users a lot, you should use this option.
  • OnDemand: Similar to the PerRequest option, a user is loaded to memory when it is needed by a request. However, the user object is kept in memory even after the request ends so that it can be reused for subsequent requests. Users are only invalidated when any change is made to the Identify database. This option is best use for installations that have large amount of users but only minority of them are used frequently.
  • All: All users are cached in memory which is best use for installations that have small amount of users and user data isn’t change very frequently. Definition of small depends on hardware configuration of the Identify server. A 4-core with 8GB of memory can easily handle thousands of users.

Do not map user logins to local store

When you set up Identify as a proxy that makes use of upstream IdPs for logging in, consider if you need to map login users to local user store. If the mapping isn’t needed, you should go to upstream IdP’s authentication connection page and tick on the “Do not map user logins to store” setting.

Certificate revocation check

Disabling certificate revocation check for every requests can help speed up token issuances.

Load test result

You can find the latest load rest result for best-case scenario at here. Please note that the result may vary depending on a number of settings as well as what claim transformation rules you configure for your installation.

We need to run another test with revocation check on to find out how much it impacts overall performance.