Home

A/B testing is widely used to determine the impact of new ideas and to decide on whether something is put in production in the long-term. Causal inference focuses on drawing conclusions based on observational data. Especially analyzing panel data (a data set with a unit and time dimension) can be challenging. For a general introduction into causal inference, we highly recommend having a look at the open-source book: Causal Inference for The Brave and True.

Installation

For the latest stable release please use the official Python package manager:

pip install -U azcausal

Estimators

azcausal is a toolkit for causal inference in general and provides implementations of well-known and widely used causal inference methods (e.g. DID, SDID).

Estimator	Reference
Difference in Difference (DID)
Synthetic Difference in Difference (SDID)	[AAH+21]
Synthetic Nearest Neighbor (SNN)	[ADSS21]

Features

Besides the estimator itself, error estimation techniques (e.g. Bootstrap, Placebo, JackKnife) attach a confidence level to the impact predictions. The result of an impact estimation in azcausal can look as follows:

╭──────────────────────────────────────────────────────────────────────────────╮
|                               CaliforniaProp99                               |
├==============================================================================┤
|                                    Panel                                     |
|  Time Periods: 31 (19/12)                                  total (pre/post)  |
|  Units: 39 (38/1)                                       total (contr/treat)  |
├──────────────────────────────────────────────────────────────────────────────┤
|                                     ATT                                      |
|  Effect (±SE): -15.60 (±7.7087)                                              |
|  Confidence Interval (95%): [-30.71 , -0.495030]                        (-)  |
|  Observed: 60.35                                                             |
|  Counter Factual: 75.95                                                      |
├──────────────────────────────────────────────────────────────────────────────┤
|                                  Percentage                                  |
|  Effect (±SE): -20.54 (±10.15)                                               |
|  Confidence Interval (95%): [-40.44 , -0.651752]                        (-)  |
|  Observed: 79.46                                                             |
|  Counter Factual: 100.00                                                     |
├──────────────────────────────────────────────────────────────────────────────┤
|                                  Cumulative                                  |
|  Effect (±SE): -187.25 (±92.50)                                              |
|  Confidence Interval (95%): [-368.55 , -5.9404]                         (-)  |
|  Observed: 724.20                                                            |
|  Counter Factual: 911.45                                                     |
╰──────────────────────────────────────────────────────────────────────────────╯

Moreover azcausal supports the visualization of results:

Contact

Feel free to contact me if you have any questions:

Julian Blank (blankjul [at] amazon.com)

Amazon.com

Applied Scientist, Amazon

410 Terry Ave N, Seattle 98109, WA.