If an Asteroid was to strike the Earth, would it affect noticeably the Earth's rotation, and if so, how large would this Asteroid have to be?
To have a noticeable effect the impactor needs to be BIG.
Most questions about "what would happen if ... hits" can be answered by the "Earth impact effects program" (http://impact.ese.ic.ac.uk/)
Here are calculations for a 100km stony asteroid...
A brute like this would have a good chance of wiping out most complex life on the planet. There has been nothing like this in the last 4 billion years (or so)... It could cause the length of the day to change by "up to 2.42 seconds"
As gerrit said, it would be the last of our worries.