> Can you give some examples and expand on this? I realize it's speculative but I'd like to hear your speculations :)

That's a big topic, probably something you could write many papers about, or a book. I probably also overstated things a bit. But it does come up in important ways.

I don't know I had anything in particular in mind although for example:

Defining a behavioral or psychological "placebo" or control becomes very complex really quickly. There's ways to approach it, such as a waitlist control, or a psychoeducational control (where clients get education but not therapy per se), or even, say, a pharmacological control for a psychotherapy study, but none of them are quite the same, and there's really no way to do true blinding. I think for a long time, people would just try to import this notion of a waitlist control or something without tackling the question of "what's an appropriate control", or trying to decide it if it's even possible. I think some of these issues have resolved but I suspect that approaching RCTs without trying to emulate drug trials so closely might have resulted in more progress faster, by honing in on what are particular components of controls versus therapies.

Another example I've discussed with my colleagues is related to our lack of understanding of the nature of psychological interventions. So for example, with a medication, it's pretty clear for the most part what the molecules involved are, their proportions, and so forth, even if we don't understand mechanisms well. But with something like, say, CBT, even if you accept its superiority in terms of efficacy (which isn't really the case), where are the boundaries? What exactly is a "cognition" and what components of it are key to understanding progress or lack thereof? A lot of psychological variables have boundaries that are fuzzy to us, and if you think about it, the notion of causality itself is a little murky (what does it mean to say that a cognitive bias "causes" negative mood?). There's some philosophical and scientific answers to these things but in general we just kind of take them for granted, and I think some of it is associated with a very operational, protocol-driven approach to intervention research influenced by other areas of medicine.

> In the case of psychotherapy my feeling is that there's an additional complication, which is that efficacy depends not only on what (is it psychotherapy B1, B2, etc.) but also on who - because efficacy has also to do with the quality of relationship between therapist and client, this varies a lot, and it is not a function of modality*. Do you agree? and if yes, what methods do you think would be best suited to studying this?

Yes there's a lot of evidence for client, therapist, and relationship factors. The nature or quality of the therapeutic relationship is one of the "common factors" posited to be shared across different therapies that have demonstrated efficacy, in the sense that you can define it for a variety of types of therapies and it predicts outcomes.

As for how to study it in a "matching" sense is tricky. It's easier to identify background characteristics of clients and therapists that predict outcomes, or to predict that, say, a given therapist is associated with better outcomes without understanding why, and you can ask clients and therapists basically some variant of "how comfortable do you feel? how is this going?" and it predicts outcomes well, but in terms of matching clients and therapists a priori is less certain. It's a bit like predicting dating outcomes, which are also tricky.

I actually have some hope for the use of LLMs and related methods in this space, with videos and whatnot. My guess is ideally there would be some kind of intake assessment of some sort; traditionally these are aimed at assessing problem areas for a client and strengths, but shifting that more to finding out who they might work well with, or what approach might work, probably would show a lot of benefits. Kind of like speed dating but maybe instead with some standard initial contact with therapy method and therapist match in mind (maybe even in some kind of online videorecording format?).

It may be there's some inherent level of unpredictability in the sense that clients don't always disclose everything intially, or aren't aware of what is relevant, so don't provide in that regard, or circumstances shift over the course of therapy (e.g., a relationship goes sour, or improves over the course of therapy) that becomes critical to the therapy match process itself. There's a kind of dynamic, self-organizing quality in that way.

I also think in general there probably isn't enough research into why clients feel like things aren't working when they feel that way, and trying to rectify it (or conversely, when a therapist is frustrated about something with a client's progress). A study to do that correctly would be really expensive and necessitate having the right diversity of therapists. Therapy studies also often tend to be organized around grand therapeutic approaches too, like CBT versus psychodynamic versus interpersonal, and less focused on specific relational questions like "what are problem areas identified by a client in therapy, with the thereapeutic relationship". It certainly happens, but probably not as often as it should.

Belated thank you for taking the time to write this!