Unless you control the ad platform, you can't make a 'no ads' control group; if you want to track the user, you've got to show them some ad; so an unrelated PSA is an ok option.

Time doesn't work very well. There are lots of reasons why you will see different behavior at different times, so it really only works if you are trying to check something with a very large effect size.

Much better is to run your control group and experiment group in parallel, like I described above.

Is your objection that showing ads for something irrelevant could affect whether the user converts?

Here's a question I have about this experiment.

So the experimental setup is: you create an ad for your product, you create an ad for your charity, then you compare the populations of people who click on your ad for your product with the population of people who click on your ad for the charity and see if there is any difference in conversion rates between the two populations.

How do you ensure that the people who see/click on your product ad aren't already a population more likely to convert to your product than the population of people who see/click on your charity ad? Sure, you can target the same groups of people - but that only goes so far, the ML algorithms backing the ad selection process will still preferentially show your charity ad to people likely to click on charity ads and show your product ad to people likely to click on your product ad.

I am unfamiliar with how these experiments work on the advertiser side. Incrementality is easy to measure on the platform side if advertisers report conversion metrics to you.

ML messing with you is an important consideration when you are trying to run this sort of experiment on a platform that you don't understand very well. The most simplest reliable way to run this experiment is to give the ad network an HTML creative that looks like:

    <script>
    var treatment = readTreatmentFromCookie();
    if (!treatment) {
      treatment = Math.random() < 0.5 ? CONTROL : EXPERIMENT;
      writeTreatmentToCookie(treatment);
    }

    if (treatment === EXPERIMENT) {
      showAd(PRODUCT);
    } else {
      showAd(CHARITY);
    }
    </script>

The creative is opaque to the network, which means it's not going to be able to do anything fancy like you're describing.

Many networks offer the ability to run a fully supported incrementality study, where they effectively use one company's ads as a control for another's, but this is a version of an experiment that you can run even if you don't trust the ad network at all.

People's behavior changes drastically with time. Showing no ad for your product seems like a sufficient control?

But then you have no control measure in most platforms, as you cannot track them?

Attribution is not my area of expertise. But could you not track them to see if they convert? This is possible both on the advertiser side and the platform side, provided the advertiser reports conversions to the platform.

You'd want to flag whether and how you captured them.

If you have offline channels directing you to a website as well, like television or print, it can get messy unless architected properly.