grammatically in a sentence

Strictly speaking, this sentence is grammatically incorrect.

Sometimes things can be not perfect grammatically, but have a much better rhythm.