Build Your Own SDK

This guide is meant for library authors looking to build a GrowthBook SDK in a currently unsupported language.

GrowthBook SDKs are very simple and do not interact with the filesystem or network. Because of this, they can often be kept to under 1000 lines of code.

All libraries should follow this specification as closely as the language permits to maintain consistency.

Data structures

Here are a number of important data structures in GrowthBook SDKs, listed alphabetically.

Attributes

Attributes are an arbitrary JSON object containing user and request attributes. Here's an example:

{
  "id": "123",
  "anonId": "abcdef",
  "company": "growthbook",
  "url": "/pricing",
  "country": "US",
  "browser": "firefox",
  "age": 25,
  "beta": true,
  "account": {
    "plan": "team",
    "seats": 10
  }
}

BucketRange

A tuple that describes a range of the numberline between 0 and 1.

The tuple has 2 parts, both floats - the start of the range and the end. For example:

[0.3, 0.7];

Condition

A Condition is evaluated against Attributes and used to target features/experiments to specific users.

The syntax is inspired by MongoDB queries. Here is an example:

{
  "country": "US",
  "browser": {
    "$in": ["firefox", "chrome"]
  },
  "email": {
    "$not": {
      "$regex": "@gmail.com$"
    }
  }
}

Context

Context object passed into the GrowthBook constructor. Has a number of optional properties:

• enabled (boolean) - Switch to globally disable all experiments. Default true.
• attributes (Attributes) - Map of user attributes that are used to assign variations
• url (string) - The URL of the current page
• features (FeatureMap) - Feature definitions (usually pulled from an API or cache)
• forcedVariations (ForcedVariationsMap) - Force specific experiments to always assign a specific variation (used for QA)
• qaMode (boolean) - If true, random assignment is disabled and only explicitly forced variations are used.
• trackingCallback (TrackingCallback) - A function that takes experiment and result as arguments.

Experiment

Defines a single Experiment. Has a number of properties:

• key (string) - The globally unique identifier for the experiment
• variations (any[]) - The different variations to choose between
• weights (number[]) - How to weight traffic between variations. Must add to 1.
• active (boolean) - If set to false, always return the control (first variation)
• coverage (number) - What percent of users should be included in the experiment (between 0 and 1, inclusive)
• condition (Condition) - Optional targeting condition
• namespace (Namespace) - Adds the experiment to a namespace
• force (number) - All users included in the experiment will be forced into the specific variation index
• hashAttribute (string) - What user attribute should be used to assign variations (defaults to id)

The only required properties are key and variations. Everything else is optional.

ExperimentResult

The result of running an Experiment given a specific Context

• inExperiment (boolean) - Whether or not the user is part of the experiment
• variationId (int) - The array index of the assigned variation
• value (any) - The array value of the assigned variation
• hashUsed (boolean) - If a hash was used to assign a variation
• hashAttribute (string) - The user attribute used to assign a variation
• hashValue (string) - The value of that attribute

The variationId and value should always be set, even when inExperiment is false.

The hashAttribute and hashValue should always be set, even when hashUsed is false.

Feature

A Feature object consists of a default value plus rules that can override the default.

• defaultValue (any) - The default value (should use null if not specified)
• rules (FeatureRule[]) - Array of FeatureRule objects that determine when and how the defaultValue gets overridden

FeatureMap

A hash or map of Feature objects. Keys are string ids for the features. Values are Feature objects. For example:

{
  "feature-1": {
    "defaultValue": false
  },
  "my_other_feature": {
    "defaultValue": 1,
    "rules": [
      {
        "force": 2
      }
    ]
  }
}

FeatureResult

The result of evaluating a Feature. Has a number of properties:

• value (any) - The assigned value of the feature
• on (boolean) - The assigned value cast to a boolean
• off (boolean) - The assigned value cast to a boolean and then negated
• source (enum) - One of "unknownFeature", "defaultValue", "force", or "experiment"
• experiment (Experiment or null) - When source is "experiment", this will be an Experiment object
• experimentResult (ExperimentResult or null) - When source is "experiment", this will be an ExperimentResult object

FeatureRule

Overrides the defaultValue of a Feature. Has a number of optional properties

• condition (Condition) - Optional targeting condition
• coverage (number) - What percent of users should be included in the experiment (between 0 and 1, inclusive)
• force (any) - Immediately force a specific value (ignore every other option besides condition and coverage)
• variations (any[]) - Run an experiment (A/B test) and randomly choose between these variations
• key (string) - The globally unique tracking key for the experiment (default to the feature key)
• weights (number[]) - How to weight traffic between variations. Must add to 1.
• namespace (Namespace) - Adds the experiment to a namespace
• hashAttribute (string) - What user attribute should be used to assign variations (defaults to id)

ForcedVariationsMap

A hash or map that forces an Experiment to always assign a specific variation. Useful for QA.

Keys are the experiment key, values are the array index of the variation. For example:

{
  "my-test": 0,
  "other-test": 1
}

Namespace

A tuple that specifies what part of a namespace an experiment includes. If two experiments are in the same namespace and their ranges don't overlap, they wil be mutually exclusive.

The tuple has 3 parts:

1. The namespace id (string)
2. The beginning of the range (float, between 0 and 1)
3. The end of the range (float, between 0 and 1)

For example:

["namespace1", 0, 0.5];

TrackingCallback

A callback function that is executed every time a user is included in an Experiment. Here's an example:

function track(experiment, result) {
  analytics.track("Experiment Viewed", {
    experimentId: experiment.key,
    variationId: result.variationId,
  });
}

Helper Functions

There are some helper functions which are used a few times throughout the SDK.

hash(str: string): float

Hashes a string to a float between 0 and 1.

Uses the simple Fowler–Noll–Vo algorithm, specifically fnv32a. An implementation of this is available in most languages already, and if not it's only a few lines of code to implement yourself.

fnv32a returns an integer, so we convert that to a float using a modulus:

n = fnv32a(str);
return (n % 1000) / 1000;

Note: It's important to use the exact hashing strategy outlined here so all SDKs behave identically.

inNamespace(userId: string, namespace: Namespace): boolean

This checks if a userId is within an experiment namespace or not.

The namespace argument is a tuple with 3 parts: id (string), start (float), and end (float).

1. Hash the userId and namespace name with two underscores as a delimiter

   n = hash(userId + "__" + namespace[0]);
   

2. Return if hash is greater than (inclusive) the namespace start and less than (exclusive) the namespace end:

   return n >= namespace[1] && n < namespace[2];
   

getEqualWeights(numVariations: integer): float[]

Returns an array of floats with numVariations items that are all equal and sum to 1. For example, getEqualWeights(2) would return [0.5, 0.5].

It's ok if the sum is slightly off due to rounding. So a sum of 0.9999999 is fine for example.

1. If numVariations is less than 1, return empty array
2. Create array with a length of numVariations
3. Fill the array with 1.0/numVariations and return

getBucketRanges(numVariations: integer, coverage: float, weights: float[]): BucketRange[]

This converts and experiment's coverage and variation weights into an array of bucket ranges.

numVariations is an integer, coverage is a float, and weights is an array of floats.

1. Clamp the value of coverage to between 0 and 1 inclusive.

   if (coverage < 0) coverage = 0;
   if (coverage > 1) coverage = 1;
   

2. Default to equal weights if the weights don't match the number of variations.

   if (weights.length != numVariations) {
     weights = getEqualWeights(numVariations);
   }
   

3. Default to equal weights if the sum is not equal 1 (or close enough when rounding errors are factored in):

   if (sum(weights) < 0.99 || sum(weights) > 1.01) {
     weights = getEqualWeights(numVariations);
   }
   

4. Convert weights to ranges and return

   cumulative = 0;
   ranges = [];
   
   for (w in weights) {
     start = cumulative;
     cumulative += w;
     ranges.push([start, start + coverage * w]);
   }
   
   return ranges;
   

Some examples:

• getBucketRanges(2, 1, [0.5, 0.5]) -> [[0, 0.5], [0.5, 1]]
• getBucketRanges(2, 0.5, [0.4, 0.6]) -> [[0, 0.2], [0.4, 0.7]]

chooseVariation(n: float, ranges: BucketRange[]): integer

Given a hash and bucket ranges, assign one of the bucket ranges.

1. Loop through ranges
   1. If n is within the range, return the range index

      if (n >= ranges[i][0] && n < ranges[i][1]) {
        return i;
      }
      

2. Return -1 if it makes it through the whole ranges array without returning

getQueryStringOverride(id: string, url: string, numVariations: integer): null|integer

This checks if an experiment variation is being forced via a URL query string. This may not be applicable for all SDKs (e.g. mobile).

As an example, if the id is my-test and url is http://localhost/?my-test=1, you would return 1.

If possible, you should use a proper URL parsing library vs relying on simple regexes.

Return null if any of these are true:

• There is no querystring
• The id is not a key in the querystring
• The variation is not an integer
• The variation is less than 0 or greater than or equal to numVariations

Evaluating Conditions

In addition to the helper functions above, there are a number of methods related to evaluating targeting conditions.

There is only one public method evalCondition and everything else is a private helper function.

public evalCondition(attributes: Attributes, condition: Condition): boolean

This is the main function used to evaluate a condition.

1. If condition has a key $or, return evalOr(attributes, condition["$or"])
2. If condition has a key $nor, return !evalOr(attributes, condition["$nor"])
3. If condition has a key $and, return evalAnd(attributes, condition["$and"])
4. If condition has a key $not, return !evalCondition(attributes, condition["$not"])
5. Loop through the condition key/value pairs
   1. If evalConditionValue(value, getPath(attributes, key)) is false, break out of loop and return false
6. Return true

private evalOr(attributes: Attributes, conditions: Condition[]): boolean

conditions is an array of Condition objects

1. If conditions is empty, return true
2. Loop through conditions
   1. If evalCondition(attributes, conditions[i]) is true, break out of the loop and return true
3. Return false

private evalAnd(attributes: Attributes, conditions: Condition[]): boolean

conditions is an array of Condition objects

1. Loop through conditions
   1. If evalCondition(attributes, conditions[i]) is false, break out of the loop and return false
2. Return true

private isOperatorObject(obj): boolean

This accepts a parsed JSON object as input and returns true if every key in the object starts with $.

• {"$gt": 1} -> true
• {"$gt": 1, "$lt": 10} -> true
• {"foo": "bar"} -> false
• {"$gt": 1, "foo": "bar"} -> false

If the object is empty and has no keys, this should also return true.

private getType(attributeValue): string

This returns the data type of the passed in argument.

The valid types to return are:

• string
• number
• boolean
• array
• object
• null
• undefined
• unknown

The difference between null and undefined can be illustrated as follows:

obj = JSON.parse('{"foo": null}');

getType(obj["foo"]); // null

getType(obj["bar"]); // undefined

The value unknown is there just in case you can't figure out the data type for whatever reason. It will never be used in most implementations.

private getPath(attributes: Attributes, path: string): any

Given attributes and a dot-separated path string, return the value at that path (or null/undefined if the path doesn't exist)

Given the input:

{
  "name": "john",
  "job": {
    "title": "developer"
  }
}

It should return:

• getPath(input, "name") -> "john"
• getPath(input, "job.title") -> "developer"
• getPath(input, "job.company") -> null or undefined

private evalConditionValue(conditionValue, attributeValue): boolean

1. If conditionValue is an object and isOperatorObject(conditionValue) is true
   1. Loop over each key/value pair
      1. If evalOperatorCondition(key, attributeValue, value) is false, return false
   2. Return true
2. Else, do a deep comparison between attributeValue and conditionValue. Return true if equal, false if not.

private elemMatch(condition, attributeValue): boolean

This checks if attributeValue is an array, and if so at least one of the array items must match the condition

1. If attributeValue is not an array, return false
2. Loop through items in attributeValue
   1. If isOperatorObject(condition)
      1. If evalConditionValue(condition, item), break out of loop and return true
   2. Else if evalCondition(item, condition), break out of loop and return true
3. Return false

private evalOperatorCondition(operator, attributeValue, conditionValue)

This function is just a case statement that handles all the possible operators

There are basic comparison operators in the form attributeValue {op} conditionValue:

• $eq ==
• $ne != (not equals)
• $lt <
• $lte <=
• $gt >
• $gte >=
• $regex ~ (regex match)

There are 2 operators where conditionValue is an array:

• $in
  1. Return true if attributeValue in the conditionValue array, false otherwise
• $nin
  1. Return false if attributeValue in the conditionValue array, true otherwise

There are 2 operators where attributeValue is an array:

• $elemMatch
  1. Return elemMatch(conditionValue, attributeValue)
• $size
  1. If attributeValue is not an array, return false
  2. Return evalConditionValue(conditionValue, attributeValue.length)

There is 1 operator where both attributeValue and conditionValue are arrays:

• $all
  1. If attributeValue is not an array, return false
  2. Loop through conditionValue array
     1. If none of the elements in the attributeValue array pass evalConditionValue(conditionValue[i], attributeValue[j]), return false
  3. Return true

There are 3 other operators:

• $exists
  1. If conditionValue is false, return true if attributeValue is null or undefined
  2. Else, return true if attributeValue is NOT null or undefined
  3. Return false by default
• $type
  1. Return getType(attributeValue) == conditionValue
• $not
  1. Return !evalConditionValue(conditionValue, attributeValue)

If operator doesn't match any of these, return false and potentially log the error for debug purposes.

GrowthBook Class

The GrowthBook class is the main export of the SDK.

constructor

The constructor takes a Context object and stores the properties for later. Nothing else needs to be done during initialization.

The main export of the libraries is a simple GrowthBook wrapper class that takes a Context object in the constructor.

This class has a few helper methods as well as 2 main public methods - feature and run.

Getters and Setters

There should be simple getters and setters for a few of the context properties:

• attributes
• features
• forcedVariations
• url
• enabled

private getFeatureResult(value, source, experiment, experimentResult): FeatureResult

This is a helper method to create a FeatureResult object. The first two arguments, value, and source are required. The last two, experiment and experimentResult are optional and should default to null.

Besides the passed-in arguments, there are two derived values - on and off, which are just the value cast to booleans.

value can be any JSON type. Only the following values are considered to be "falsy":

• null
• false
• ""
• 0

Everything else is considered "truthy", including empty arrays and objects.

If value is "truthy", then on should be true and off should be false. If the value is "falsy", then they should take opposite values.

private getExperimentResult(experiment, variationIndex, hashUsed): ExperimentResult

This is a helper method to create an ExperimentResult object. The arguments are:

• experiment - Experiment object (required)
• variationIndex - The assigned variation index (optional, default to -1)
• hashUsed - Whether or not the hash was used to assign a variation (optional, default to false)

The method is pretty simple:

1. Handle case when user is not in the experiment (e.g. variationIndex = -1)

   // By default, assume everyone is in the experiment
   let inExperiment = true;
   // If the variation is invalid, use the baseline and set the inExperiment flag to false
   if (variationIndex < 0 || variationIndex >= experiment.variations.length) {
     variationIndex = 0;
     inExperiment = false;
   }
   

2. Get the hashAttribute and hashValue

   hashAttribute = experiment.hashAttribute || "id";
   hashValue = context.attributes[hashAttribute] || "";
   

3. Return

   return {
     inExperiment: inExperiment,
     hashUsed: hashUsed,
     variationId: variationIndex,
     value: experiment.variations[variationIndex],
     hashAttribute: hashAttribute,
     hashvalue: hashValue,
   };
   

public evalFeature(key: string): FeatureResult

The evalFeature method takes a single string argument, which is the unique identifier for the feature and returns a FeatureResult object.

growthbook = new GrowthBook(context);
myFeature = growthbook.evalFeature("my-feature");

There are a few ordered steps to evaluate a feature

1. If the key doesn't exist in context.features
   1. Return getFeatureResult(null, "unknownFeature")
2. Loop through the feature rules (if any)
   1. If the rule has a condition
      1. If evalCondition(context.attributes, rule.condition) is false, skip this rule and continue to the next one
   2. If rule.force is set
      1. If rule.coverage is set
         1. Get the value of the hashAttribute (context.attributes[rule.hashAttribute || "id"]) and if empty, skip the rule
         2. Hash the value

            n = hash(hashValue + featureKey);
            

         3. If the hash is greater than rule.coverage, skip the rule
      2. Return getFeatureResult(rule.force, "force")
   3. Otherwise, convert the rule to an Experiment object

      const exp = {
        variations: rule.variations,
        key: rule.key || featureKey,
      };
      if (rule.coverage) exp.coverage = rule.coverage;
      if (rule.weights) exp.weights = rule.weights;
      if (rule.hashAttribute) exp.hashAttribute = rule.hashAttribute;
      if (rule.namespace) exp.namespace = rule.namespace;
      

   4. Run the experiment

      result = run(exp);
      

   5. If result.inExperiment is false, skip this rule and continue to the next one
   6. Otherwise, return

      return getFeatureResult(result.value, "experiment", exp, result);
      

3. Return getFeatureResult(feature.defaultValue || null, "defaultValue")

public run(experiment: Experiment): ExperimentResult

The run method takes an Experiment object and returns an ExperimentResult.

There are a bunch of ordered steps to run an experiment:

1. If experiment.variations has fewer than 2 variations, return getExperimentResult(experiment)
2. If context.enabled is false, return getExperimentResult(experiment)
3. If context.url exists

   qsOverride = getQueryStringOverride(experiment.key, context.url);
   if (qsOverride != null) {
     return getExperimentResult(experiment, qsOverride);
   }
   

4. Return if forced via context

   if (experiment.key in context.forcedVariations) {
     return getExperimentResult(
       experiment,
       context.forcedVariations[experiment.key]
     );
   }
   

5. If experiment.active is set to false, return getExperimentResult(experiment)
6. Get the user hash value and return if empty

   hashAttribute = experiment.hashAttribute || "id";
   hashValue = context.attributes[hashAttribute] || "";
   if (hashValue == "") {
     return getExperimentResult(experiment);
   }
   

7. If experiment.namespace is set, return if not in range

   if (!inNamespace(hashValue, experiment.namespace)) {
     return getExperimentResult(experiment);
   }
   

8. If experiment.condition is set return if it evaluates to false

   if (!evalCondition(context.attributes, experiment.condition)) {
     return getExperimentResult(experiment);
   }
   

9. Calculate bucket ranges for the variations and choose one

   ranges = getBucketRanges(
     experiment.variations.length,
     experiment.converage ?? 1,
     experiment.weights ?? []
   );
   n = hash(hashValue + experiment.key);
   assigned = chooseVariation(n, ranges);
   

10. If assigned == -1, return getExperimentResult(experiment)
11. If experiment has a forced variation, return

    if ("force" in experiment) {
      return getExperimentResult(experiment, experiment.force);
    }
    

12. If context.qaMode, return getExperimentResult(experiment)
13. Build the result object

    result = getExperimentResult(experiment, assigned, true);
    

14. Fire context.trackingCallback if set and the combination of hashAttribute, hashValue, experiment.key, and variationId has not been tracked before
15. Return result

Feature helper methods

There are 3 tiny helper methods that wrap evalFeature for a better developer experience:

public isOn(key) {
  return this.evalFeature(key).on
}

public isOff(key) {
  return this.evalFeature(key).off
}

public getFeatureValue(key, fallback) {
  value = this.evalFeature(key).value
  return value === null ? fallback : value
}

For strongly typed languages, you can use generics (if supported) for getFeatureValue and coerce the return value to always match the data type of fallback. If generics are not supported, you can use type-specific versions of the function such as getFeatureValueAsString.

Type Hinting

Most languages have some sort of strong typing support, whether in the language itself or via annotations. This helps to reduce errors and is highly encouraged for SDKs.

If possible, use generics to type the return value. For example, if experiment.variations is type T[], then result.value should be type T. Or, if the fallback of getFeatureValue is type string, the return type should also be type string.

Handling Errors

The general rule is to be strict in development and lenient in production.

You can throw exceptions in development, but someone's production app should never crash because of a call to growthbook.evalFeature or growthbook.run.

For the below edge cases in production, just act as if the problematic property didn't exist and ignore errors:

• experiment.weights is a different length from experiment.variations
• experiment.weights adds up to something other than 1
• experiment.coverage or feature.coverage is greater than 1 or less than 0
• context.trackingCallback throws an error
• URL querystring specifies an invalid variation index

For the below edge cases in production, the experiment should be disabled (everyone gets assigned variation 0):

• experiment.coverage is less than 0
• experiment.force specifies an invalid variation index
• context.forcedVariations specifies an invalid variation index
• experiment.hashAttribute is an empty string

Subscriptions

Sometimes it's useful to be able to "subscribe" to a GrowthBook instance and be alerted every time growthbook.run is called. This is different from the tracking callback since it also fires when a user is not included in an experiment.

growthbook.subscribe(function (experiment, result) {
  // do something
});

It's best to only re-fire the callbacks for an experiment if the result has changed. That means either the inExperiment flag has changed or the variationId has changed.

If it makes sense for your language, this function should return an "unsubscriber". A simple callback that removes the subscription.

unsubscriber = growthbook.subscribe(...)
unsubscriber()

In addition to subscriptions you may also want to expose a growthbook.getAllResults method that returns a map of the latest results indexed by experiment key.

Memory Management

Subscriptions and tracking calls require storing references to many objects and functions. If it makes sense for your language, libraries should provide a growthbook.destroy method to remove all of these references and release their memory.

Tests

We strive to have 100% test coverage for all of our SDKs.

There is a language-agnostic test suite stored as a JSON file packages/sdk-js/test/cases.json with more than 200 unit tests. This extensively tests all of the public methods mentioned above.

The cases.json file is an object. The keys are the function being tested, and the values are arrays of test cases. The test case arrays structure is different for each function and listed below:

• hash
  • value to hash (string)
  • expected result (float)
• getBucketRange
  • Name of the test case (string)
  • Arguments array ([numVariations, coverage, weights or null])
  • expected result
• chooseVariation
  • name of the test case (string)
  • n (hash)
  • bucket ranges
  • expected result
• getQueryStringOverride
  • name of the test case (string)
  • experiment key
  • url
  • numVariations
  • expected result
• inNamespace
  • name of the test case (string)
  • id
  • namespace
  • expected result
• getEqualWeights
  • numVariations
  • expected result (weights rounded to 8 decimal places)
• evalCondition
  • name of the test case (string)
  • condition
  • attributes
  • expected return value (boolean)
• feature (evalFeature)
  • name of the test case (string)
  • context passed into GrowthBook constructor
  • name of the feature (string)
  • expected result
• run
  • name of the test case (string)
  • context passed into GrowthBook constructor
  • experiment object
  • expected value
  • inExperiment (boolean)
  • hashUsed (boolean)

In addition to the above, you should write custom test cases for things like event subscriptions, tracking callbacks, getters/setters, etc. that are more language-specific.

Getting Help

Join our Slack community if you need help or want to chat. We're also happy to hop on a call and do some pair programming.

Attribution

Open a GitHub issue with a link to your project and we'll make sure we add it to our docs and give you proper credit for your hard work.