The Chidori CAN be used without the speed. We see stone disintegrate stone on contact, so imagine what will happen to human flesh. However, the Chidori's INTENDED purpose is to pierce hardened targets. Hence, the speed needed to maximize its destructive power. A very fast Taijutsu user probably wouldn't pick the straight ahead charge method of fighting. They use their speed to maneuver around their opponents so they attack from the back or the sides. They depend on changing their direction quickly, not gradually building up speed in a straight line charge.