Manipulating the polygon size is one way to approach it, but it’s probably more complicated. I think it would be better to divide the tongue into differing poly groups (long strips probably), and apply the nano to each group in turn rather than all at once, scaling and rotating as necessary. Simply set the nano brush polygon target to the appropriate setting for the polygons you want to target, rather than “all polygons”.
Once the nanos are all placed, you can also do some adjustment by hand, by masking the underlying tongue shape, and running an inflat or magnify brush at very low z intensity over sections of the spike hooks.
The new VDM (vector displacement alpha brushes) may have an application here as well. Notice in the following video how he uses a VDM brush with lazy mouse to apply a long string of protrusions, which can be varied with brush size setting: