It will depend on the flatness of the existing substrate. If the substrate is flat under a 2m edge, you will get away with a smaller notch, but if the substrate has discrepancies then you will need a larger notch trowel to ensure a solid bed fix.
In order to support traffic weight, the adhessive for both insulation boards and tiles must be applied with a 10mm notched trowel. (Bal recommendations)
I have used just about every size trowel I have at one stage or another :lol: it depends on the state of the floor. I usually use a 6mm for flat floors and then go up in size depending on how bad it gets. The important thing is to leave no voids under the boards. I back fill the boards and really push/twist/slide the boards down.