You are not the only one whom sees a difference but remember that the statistics are skewed. Fewer people are awake early to see, or photograph, the sunrise while more people are awake to see the sunset.
People and heating of the Earth are factors explaining the difference in the atmosphere through which the sunlight is filtered, that affects the appearance of the light.
It's well explained at our Photo.SE site:
Most of the physical reasons (I can think of) probably do favor sunsets as well. During the day, the sun is heating the air and the ground. This leads to air movement that tends to pick up particulates (e.g., dust, smoke, etc.) and keep it in the air. It also leads to water evaporating (relatively) quickly from anything like lakes and rivers in the area. In addition, quite a few things that people do (e.g., driving) tends to happen primarily during the day, so it "pushes" more pollution into the air during the day. Thus, at the end of the day, the light is shining through air that carries more particles, which almost inevitably does more to diffuse the light.
During the night, there's a lot less pollution being put into the air, and the lack of heating means the air tends to be calmer as well, so by morning many of those particulates will have simply fallen out of the air, and the humidity either formed into clouds or precipitated. The clearer air does less to diffuse the light, so the sunrise isn't as spectacular.
A more scientific, but not complicated, explanation is offered at Wikipedia's "Sky" webpage. The heat affects the Rayleigh scattering and the Lorenz–Mie–Debye solution or Mie scattering, that explains the differences between sunrise and the sunset - a difference in the number and movement of particles in the sky.
A somewhat related factor is explained in a Q&A over at Space.SE: "Where does space begin on planets without atmospheres?". The height of the atmosphere and its layers along with the distribution of its component gases is altered by the heating of the planet and atmospheric circulation.