Variety is the slice of life

Rethinking experiment generation

Description

Variety is an experiment variation generator. What does that mean? We can generate variable length numbers between 1 to N where N is the number of varients. We then can use this list of numbers to decide what should be enabled and disabled.

Goals

To implemented a universal algorithm based on a single unsigned int id that can be used in multiple languages.

• implemented in rust
• ffi in perl
• ffi in java
• ffi in node

Why?

Testing is hard. Testing with experiments is harder. By decoupling feature toggles logic from experiment's we can let both exist together in harmony. The second half of this work involves allowing per 'user' where user is anything calling an application (where application is a single unit of feature toggles and experiments) to decide their experience. This is useful for developer experience and replicating problems our users experience. For the most basic example imagine a we identify user with id 1234 has triggered several exceptions during their usage and want to evaluate exactly what they were doing, with our current method of experimentation this is non trivial involving headers and decoding. This project proposes a single unsigned 32 integer as an 'experiment ID' that can be encoded as a hexidecimal string for transport. This experiment ID will provide the exact same experience for any user that can provide it. With this we can replicate issues with conflicting experiments and even have fine grained control when creating integration tests.

How?

Rust! We can create a .so library allowing for every language to provide their own API's calling to our native implementation. Rust can be seen as a nicer C. To begin with we will focus on providing JNI bindings to our rust library.

Authors

Luke Collier