Perpendicular exchange bias (PEB) between [Co 0.3 nm/Pd 0.8 nm]5 multilayers and IrMn antiferromagnetic (AFM) layer is studied as functions of thickness of the interface layer and the AFM layer. It is found that increasing the thickness of a CoFe interface layer up to 2.1 nm could effectively improve the PEB. The achieved PEB field (Hbias) is more than 500 Oe. On the other hand, the coercivity (Hc) exhibits an opposite trend as a function of CoFe interface layer thickness, which might promote the integration of PEB structure with the perpendicular magnetic memory stack. It is also found that PEB is sensitive to the thickness of the AFM layer. The thickness window is only around 2 nm to achieve the largest Hbias. Moreover, for a very thin IrMn layer, a ferromagnetic nature is observed at low magnetic fields, which is likely owing to the net spins at the surface that might have been magnetized by the CoFe interface layer.