Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

ML messing with you is an important consideration when you are trying to run this sort of experiment on a platform that you don't understand very well. The most simplest reliable way to run this experiment is to give the ad network an HTML creative that looks like:

    <script>
    var treatment = readTreatmentFromCookie();
    if (!treatment) {
      treatment = Math.random() < 0.5 ? CONTROL : EXPERIMENT;
      writeTreatmentToCookie(treatment);
    }

    if (treatment === EXPERIMENT) {
      showAd(PRODUCT);
    } else {
      showAd(CHARITY);
    }
    </script>
The creative is opaque to the network, which means it's not going to be able to do anything fancy like you're describing.

Many networks offer the ability to run a fully supported incrementality study, where they effectively use one company's ads as a control for another's, but this is a version of an experiment that you can run even if you don't trust the ad network at all.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: