What you're describing is basically the "collapsed star" (Eng) or "frozen star" (Rus) interpretation of black holes that was common prior to the late mid-1960s. It was a mistake.
Suppose you are distant and stationary relative to the black hole. You will observe infalling matter asymptotically approaching the horizon, growing ever fainter as it redshifts. Does it mean that matter "clumps" around the horizon? To find out, suppose you throw yourself towards the black hole to try to catch the matter that you see. What you will find is that it fell into the black hole long ago.
In other words, the most sensible way to answer whether or not infalling matter clumps on the horizon is to look at the situation from the frame of that infalling matter. And there, it is clear: no, it does not clump, as it crosses the horizon in finite proper time. (As an aside, for a Schwarzschild black hole, falling from rest is exactly Newtonian in Schwarzschild radial coordinate and proper time.)
The "comoving viewpoint" was recognized by Oppenheimer and Snyder in 1939, but it was not until the 1960s, with the work of Zel'dovich, Novikov, et al., that it was generally recognized as truly significant in the community. In 1965, Penrose introduced conformal diagrams based on the Eddington-Finkelstein coordinates (1924/1958) that showed quite clearly that the stellar collapse is not slowed, but instead continues to a singularity. For an overview of the history of this change of viewpoint, cf. Kip Thorne, et al., The Memberane Paradigm (1986). These topics are commonly covered in many relativity textbooks.
Ok, but since it still takes an infinite amount time in the frame adapted to a stationary distant observer, does that mean that the horizon never forms in that frame? It does form: the underlying assumption in the argument that it does not would be either that the infalling matter needs to reach the center for the horizon to form or cross a pre-existing horizon to make it expand. But that assumption is simply not true.
An event horizon is defined in terms of future lightlike infinity, roughly speaking in terms of whether or not light rays escape if one waits an infinite amount of time. That means the location of the horizon at any time depends on not just what has happened, but also what will happen in the future. In the frame of the distant stationary observer, as matter falls towards the event horizon, it does slow down to asymptotically approach... but the horizon also expands to meet it. Similarly, the initial collapsing matter does not need to collapse all the way to the center for the event horizon to form.
How can the finite life-time of the Black hole due to Hawking radiation be made consistant with the infinite amount of time (future) needed for the expansion of the event horizon (in the outer time-frame)?
There's no need to: [edit]that a particular time coordinate doesn't cover the full manifold is a fault of the coordinate chart, not of spacetime[/edit]. From every event, send out an omnidirectional locus of idealized light rays. The event horizon is the boundary of the spacetime region from which none of these light rays escape to infinity. This question has an objective answer--for any given light ray, either it will escape or it won't.
An external observer would need to wait infinitely long to know for sure where the event horizon is exactly, but that's a completely different issue. With Hawking radiation, the black hole shrinks, but it doesn't change the fact that light rays from some events will fail to escape, and thus that an event horizon will exist.
Here's a Penrose diagram of a spherically collapsing star forming a black hole that subsequently evaporates:
Light rays run diagonally at ±45° on the diagram. Note that there is a region from which outgoing light rays (running diagonally lower-left to upper-right) don't escape and instead meet the $r = 0$ singularity (the bolded, undashed horizontal line). The horizon itself is the the $r = 2m$ line marked on the diagram and its extension into the star: it should actually go from the (dashed, vertical) $r = 0$ line on the left, rather than extending from the star's collapsing surface. That's because some of the (idealized, noninteracting) light rays from inside the star will also fail to escape to infinity.
Now suppose that on this diagram you draw timelike curves that stubbornly stay away from the horizon, and you insist on using a parameter along them as a time coordinate. Does the fact that you've chosen coordinates that exclude the horizon needs to be made consistent with whether or not the event horizon actually exist? The resolution is simple: if you want to talk about the horizon, stop using coordinates that exclude it.