If an Asteroid was to strike the Earth, would it affect noticeably the Earth's rotation, and if so, how large would this Asteroid have to be?
To have a noticeable effect the impactor needs to be BIG.
Most questions about "what would happen if ... hits" can be answered by the "Earth impact effects program" (http://impact.ese.ic.ac.uk/)
Here are calculations for a 100km stony asteroid...
A brute like this would have a good chance of wiping out most complex life on the planet. There has been nothing like this in the last 4 billion years (or so)... It could cause the length of the day to change by "up to 2.42 seconds"
As gerrit said, it would be the last of our worries.
An asteroid less than a kilometer wide impacting the earth could noticeably affect the rotation rate.
When an asteroid strikes the earth in an inelastic collision, momentum is conserved between the asteroid and the earth (as long as no material is ejected). That means that all the linear momentum from the asteroid is transferred into the momentum of the earth's revolution around the sun and the earth's rotation around its own axis.
The transfer of asteroid momentum into the earth's rotational angular momentum will be maximized if the asteroid's trajectory lies within the equatorial plane and strikes along the equator at a shallow angle (similar to spinning the cue ball in the game of billiards). For the below calculations, I choose asteroid trajectory, speed, and density to minimize the size of the asteroid necessary to alter the earth's rotation rate.
The rotation rate of the earth is known to within $\omega =1e-13$ radians per second https://en.wikipedia.org/wiki/Earth%27s_rotation . So we would notice via GPS measurements if the rate changed by more than that.
The moment of inertia of a solid sphere is $I=2/5MR^2$ where $M$ and $R$ are the mass and radius of the earth. Using wikipedia values, $I$ is about $9.3e37kgm^2$. So the angular momentum change to noticeable slow the earth is $I\omega = 9.3e24kgm^2/s$. To translate this to linear momentum, we can divide this by the equatorial radius of the earth to get $l=1.53e18kgm/s$. This is the linear momentum an asteroid would need to impart to the earth, along the equator, tangent to the earth's surface to noticeably alter the rotation rate.
If we take a fast meteor like Oumoamoa that was going about 50km/s at 1 AU, and suppose it has an impact trajectory opposite the earth's orbit, we can add the earth's orbital speed of 30km/s to get a whopping speed of $s = 8e5m/s$ impact. Since linear momentum $l=mv$ the mass of the asteroid is $m = l/s = 1.53e18/8e5 = 1.925e12kg$.
The above calculations assume that the asteroid strikes the earth surface at zero degrees, but such a trajectory would cause the asteroid to bounce off atmosphere back into space. A steeper angle like 45 degrees would mean only half of the asteroid's linear momentum would be transferred into the earth's angular momentum, so we would need an asteroid twice as massive or $m=3.85e12kg$.
If the asteroid is a very dense $d=9000kg/m^3$ then the volume of the asteroid is $v = m/d = 4.3e8m^3$. Since volume of a sphere is $v = 4/3\pi r^3$, if we solve for the radius, we get $r=468m$. So the diameter of the asteroid is $d=2r=936m$ wide or slightly less than a kilometer!
Note: We have never had an asteroid this large strike in human history (10,000 years ago), but at least 6 have struck the earth since humans evolved (300,000 years ago). https://en.wikipedia.org/wiki/List_of_impact_craters_on_Earth