A multi-channel polarization-sensitive Mueller-matrix optical coherence tomography (OCT) was built with single-mode optical fibers in both the sample and reference arms. A new rigorous algorithm was developed to eliminate dynamically the polarization distortions caused by the sampling fiber and consequently retrieve the calibrated polarization properties of the sample. The roundtrip Jones matrix of the sampling fiber used in the algorithm was acquired from the reflecting surface of the sample for each depth scan (A scan). Both this new algorithm and the algorithm used in previous fiber-based polarization-sensitive OCT (PS-OCT) were tested with simulated data, which shows that the only parameter that can be correctly retrieved by the previous algorithm is the phase retardation. The skin of a rat was imaged with this fiber-based system.