"If you forgive the sins of any, their sins have been forgiven them; if you retain the sins of any, they have been retained."
The word for "sin" here is hamartia, and can be translated as an offense.
The word for "forgive" is aphiemi. This can be translated as forgive, or other words like remit. Remit is a word we don't use much, but means to pardon, to refrain from enforcing a punishment.
The word for "retained" is krateo. Retained is one of the literal translations so lets look at the definition of "retain". One definition is to keep in mind, to remember, or to hold in place, like a retaining wall.
I believe that this verse is not telling us that we have the power to forgive sins against God, but that we have the power to forgive offenses toward ourselves. If we forgive someone who offends us, we don't enforce the punishment, even if deserved.
The second half of the verse does not talk about forgiving, it talks about retaining. It also is talking about what we retain, not what the other person retains. I believe what this is telling us is that when we don't forgive, we retain. Those offenses are kept in us, building up until let go, until the retaining wall is torn down, releasing the offenses and the offenders.
Forgiveness is as much or more for the offendee's benefit as it is for the offender's.