I need to sweep with torsion and different diameters the active part of the dental file
but it just continues to sweep with the same diameter.
I think different diameter sweep is not possible in solid works better you go for loft with scaling your sketch as per required diameter
Not as a sweep, but you can as a loft, with a odd twist.
Perhaps something like this?